Abstract:
A system and method for a network to deconvolve mixtures of inputs that have been previously learned. In addition, the network is also able to segment the components of each input object that most contribute to its classification. The network consists of oscillatory units that can comprise amplitude and phase, and that can synchronize their dynamics, so that deconvolution is determined by the amplitude of an output layer, and segmentation by phase similarity between input and output layer units. Moreover, segmentation can be achieved even when there is considerable superposition of the inputs.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS  
       [0001]     Not Applicable.  
       STATEMENT REGARDING FEDERALLY SPONSORED-RESEARCH OR DEVELOPMENT  
       [0002]     Not Applicable.  
       INCORPORATION BY REFERENCE OF MATERIAL SUBMITTED ON A COMPACT DISC  
       [0003]     Not Applicable.  
       FIELD OF THE INVENTION  
       [0004]     The invention disclosed broadly relates to field of signal processing, and separation of source signals from a mixture of signals and more specifically to the fields of signal deconvolution.  
       BACKGROUND OF THE INVENTION  
       [0005]     An important problem described in the signal processing and neural information processing literature is that of the so-called cocktail party problem, where one would like to identify individual voices when they are mixed together. See Ch. Von der Malsburg and W. Schneider, “A Neural Cocktail Party Processor,” Biol. Cybern., 54(1):29-40 (1986). This problem has been tackled by methods such as independent component analysis (ICA). See A. J. Bell and T. J. Sejnowski, “An information-maximization approach to blind separation and blind deconvolution,”  Neural Computation,  7:1129-1159 (1995).  
         [0006]     Though techniques such as ICA can perform the separation of the signal sources, they cannot directly identify which signal source is dominant at any particular instant in time. The reason is that they are global techniques, and make use of the probability distributions of the different signal sources, which requires the extraction of global statistics. It is difficult for these techniques to provide precise local information such as which signal is dominant at an instant in time.  
         [0007]     We refer to this ability to provide local information as the ability to segment the input signal. Hence, a technique is desired which can provide signal separation or identification combined with segmentation of learning, the network is able to recognize previous inputs that it has been shown.  
         [0008]     Deconvolution and blind deconvolution (i.e. identifying the presence of specific objects in the visual field) have been extensively studied in the neural network literature. A. J. Bell and T. J. Sejnowski, supra. On the other hand, segmentation, which refers to the ability to identify the elements of the input space that uniquely contribute to each specific object (i.e. establishing a correspondence between the pixels or edges and the higher-level objects they belong to), has been attacked more effectively with non-neural approaches. S. Ullman, M. Vidal-Naquet, E. Sali E., “Visual features of intermediate complexity and their use in classification,”  Nature Neuroscience  5(7):682-7 (2002).  
         [0009]     However, inspired by experimental evidence of a role for synchronization of neural responses in a variety of motor and cognitive tasks, and in particular in perceptual recognition, Malsburg and Shneider were among the first to propose the use of synchronization to perform segmentation of a mixture of signals. C. M Gray, P. Koenig, A. K. Engel and W. Singer, “Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties,”  Nature,  338(6213):334-337 (1989); E. Rodriguez, N. George, J. P. Lachaux, J. Martinerie, B. Renault and F. J. Varela, “Perception&#39;s shadow: long-distance synchronization of human brain activity. Nature,” 397(6718):430-433 (1999); Ch. von der Malsburg and W. Schneider, “A neural cocktail-party processor,”  Biol. Cybern.,  54(1):29-40 (1986). Their model consists of a layer of excitatory units connected with lateral excitation. Each of these excitatory units receives sensory input. Furthermore, every excitatory unit is connected to a global inhibitory unit which receives excitatory inputs, and sends inhibitory signals to each of the excitatory units. Segmentation is exhibited in the form of temporal correlation amongst the activities of the different excitatory units, so that the units that are synchronized represent the same input class. Besides the need for a global inhibitory unit, this network cannot disambiguate objects with partial overlap. Ch. von der Malsburg and W. Schneider, supra. Indeed, a number of approaches derived from inherit the same shortcomings, and therefore the issue of effective segmentation by networks of synchronizing units needs to be addressed. J. Buhmann and C. Von Der Malsburg, “Sensory segmentation by neural oscillators,”  International Joint Conference on Neural Networks, Part II,  pp. 603-607 (1991), K. Chen and D. Wang and X. Liu, “Weight Adaptation and Oscillatory Correlation for Image Segmentation,”  IEEE Transactions on Neural Networks,  11(5):1106-1123 (2000), D. L. Wang and X. Liu, “Scene analysis by integrating primitive segmentation and associative memory,”  IEEE Transactions on Systems, Man, and Cybernetics, Part B,  32(3):254-268 (2002).  
         [0010]     The original network proposed by Malsburg and Shneider has been influential in advancing a theory for the use of synchrony as a solution to segmentation. J. Buhmann and C. Von Der Malsburg, “Sensory segmentation by neural oscillators,” International Joint Conference on Neural Networks, Part II, pp. 603-607 (1991). However, the specific implementation proposed in their paper has several shortcomings. Firstly, a global inhibitory neuron is required. Secondly, learning in their model requires a combination of short-term and long-term synaptic modification. Thirdly, the test cases used in their model did not involve any overlap amongst the spectral inputs to be separated. Buhman and Malsburg explicitly introduced oscillatory units into the model, but their model suffers from earlier noted shortcoming in that the presence of a global inhibitory unit is required. Buhmann and C. Von Der Malsburg, Sensory segmentation by neural oscillators. International Joint Conference on Neural Networks, Part II, pp. 603-607(1991). The subsequent work of Chen, Wang and Liu, and Wang and Liu offer enhancements of the original model, but maintains the essential aspect of utilizing a global inhibitor. K. Chen and D. Wang and X. Liu, “Weight Adaptation and Oscillatory Correlation for Image Segmentation,” IEEE Transactions on Neural Networks, 11(5):1106-1123(2000). The work of Izhikevich is mainly theoretical, and does not present any specific methodology to address the problem of segmentation. E. M. Izhikevich, “Weakly Pulse-Coupled Oscillators, FM Interactions,” Synchronization, and Oscillatory Associative Memory. IEEE Transactions on Neural Networks, 10(3):508-526 (1999). Hoppensteadt and Izhikevich illustrate their method with a single example using three inputs, and have not applied their methodology to a larger number of inputs or test cases, or addressed the segmentation problem. F. C. Hoppensteadt and E. M. Izhikevich, “Pattern Recognition Via Synchronization in Phase-Locked Loop Neural Networks,” IEEE Transactions on Neural Networks, 11(3):734 (1999). Furthermore, they raise the issue that the Hebbian learning rule they use may not be the best. The method of Sun et al requires the use of visual motion to perform segmentation, and hence is not applicable to static inputs as we have investigated. Furthermore, their scheme relies on supervised training, and uses back-propagation learning. H. Sun, L. Liu and A. Guo, “A Neurocomputational Model of Figure-Ground Discrimination and Target Tracking,” IEEE Transactions on Neural Networks, 10(4):860-884(1999).  
         [0011]     U.S. Pat. Nos. 6,236,862 B1 and 6,625,587 B1 disclose a method and apparatus for dynamically separating signal sources from a received mixture. Their method however, does not address and solve the problem of segmentation as described in the current invention, and the establishment of a correspondence between local input features and individual signal sources.  
       SUMMARY OF THE INVENTION  
       [0012]     In an embodiment of the invention, a network architecture can efficiently segment overlapping one-dimensional inputs, and can be generalized to higher dimensions. The network used in this embodiment comprises oscillatory units that each possesses an amplitude of oscillation, a frequency, and phase. Out of these properties, the amplitude and phase are the most critical for the network to exhibit the desired behavior of segmentation and deconvolution. These units are organized into multiple layers, and each unit receives feedforward, feedback, and lateral connections from other units. Furthermore, each unit is connected to other units. The different classes of connections (feedforward, feedback and lateral) affect the receiving unit in different ways. Each connection is represented by a weight, which is learnt or modified according to learning rules. Through this process of learning, the network is able to recognize previous inputs that it has been shown. This learning proceeds in a self-organized manner, i.e. the process is unsupervised.  
         [0013]     This network is also able to deconvolve mixtures of inputs that have been previously learned. In addition, the network can segment the components of each input object that most contribute to its classification. This is achieved by the ability of the units in the network can synchronize their dynamics, so that deconvolution is determined by the amplitude of an output layer, and segmentation by phase similarity between input and output layer units. Learning is unsupervised and based on a Hebbian update, and the architecture is very simple. Moreover, efficient segmentation can be achieved even when there is considerable superposition of the inputs.  
         [0014]     One embodiment overcomes the global inhibitory restriction and spreads inhibition across the entire network, which is more biologically plausible. In our model the learning is reduced to a single generic rule. The long term and short term synaptic modification, which in some embodiments is reduced to a single generic rule. Embodiments allow complete overlap, and shows that successful separation and segmentation is still possible. Other embodiments use the Hebbian rule, which is simple, and we have shown that it works extremely well.  
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0015]     FIGS.  1 A-C are a high level block diagrams showing a two-layer representation of a network according to an embodiment of the invention.  
         [0016]      FIG. 2A  is an instance of the input of an input ensemble.  
         [0017]      FIG. 2B  illustrates the behavior of the network after training.  
         [0018]      FIG. 3A  illustrates the two inputs used.  
         [0019]      FIG. 3B  is a mixture of the two inputs presented to the system.  
         [0020]      FIG. 3C  is the winner for input  1 , W 1  is the winner for input  1  and W 2  is the winner for input  2 .  
         [0021]      FIG. 4A  is the conditional probability distribution for deconvolution figures.  
         [0022]      FIG. 4B  shows average segmentation accuracy versus the dot product of the inputs selected for mixing.  
         [0023]      FIG. 5A  is the average deconvolution accuracy versus the noise level.  
         [0024]      FIG. 5B  is an average classification accuracy versus the noise level. 
     
    
     DETAILED DESCRIPTION  
       [0025]     Referring to  FIG. 1A , there is shown a block diagram of a learning (neural) network  100  according to an embodiment of the invention. The network  100  comprises a plurality of units (e.g., neurons) in an input (bottom) layer  102 , a second plurality  104  of units in an output (upper) layer, and a feedforward connection  103  to each of the second plurality of units  104 .  FIG. 1B , shows the feedback  108  connection from the output layer  104  to the input layer  102 .  FIG. 1C  shows the lateral connections  105  within the output layer  104 .  
         [0026]     The network  100  performs dynamical segmentation based on the idea that each of the network&#39;s units can be described in terms of an amplitude and a phase, and that the feedforward and feedback connections (excitatory or inhibitory) can affect the receiving unit&#39;s amplitude and phase in qualitatively different ways.  
         [0027]     The input (bottom) layer  102  receives an input from an input signal  106 . The network  100  comprises dynamical units. The amplitude output of these units is only a function of their inputs, whereas the phase is a function of the their internal frequency, and feedback interactions with an output layer  104 . An output layer  104  consists of dynamical units that receive an input from the input layer  102  through feedforward connections  108 . For these units, the amplitude and the phase are computed by integrating inputs as a function of their amplitude and their phase difference with respect to the receiving phase. The output layer  104  sends feedback to the input layer  102 , which is used to modify only the phase of the bottom layer&#39;s units as a receiving phases.  
         [0028]     The input space consists of an ensemble of vectors {x n }, n=1, . . . , M, such that x n  ε [0,1] N , |x n |=1∀n. The bottom layer  102  consists of N oscillators with amplitude r 1 ≧0, phase θ 1  ε [0, 2π], and frequency ω 1  ε [ω 1   m , ω 1   M ]; similarly the top layer consists of N oscillators described by amplitude r 2 , phase θ 2  and frequency ω 2  ε [ω 2   m ,ω 2   M ]. The bottom layer feeds forward into the top one with connection , where i ranges over top units and j over bottom ones. Similarly, the top layer feeds back into the bottom one, with connections W i   F   j  where i ranges over the bottom units, and j over the top ones. The top layer has inhibitory connections onto itself, Gij . Feed-forward and feedback connections are normalized, such that
 
 W   i   l  ε [0, 1] N   , ∥W   i   j ∥=1, where  W   i   ={W   i1   , . . . , W   iN }
 
 and f={F, B, I}. 
 
         [0029]     The network operates in two stages, learning and performance. Only during the learning stage are the feedforward and feedback connections modified, whereas the inhibitory connections are fixed throughout. During the learning stage, elements of the input ensemble are presented to the network, upon which the response of the network is dynamically computed. A unit&#39;s phase update is the result of its internal frequency, and of integrating all feedforward, inhibitory and feedback inputs, weighted by their amplitude and the receiving unit&#39;s amplitude, as well as by a non-linear function of their relative phases with respect to the receiving unit. For the amplitude update, the incoming amplitudes are weighted by a non-linear function of the relative phases, and limited by a leakage function of the receiving unit&#39;s amplitude. Qualitatively, the effect of one input unit j, where j ε[1 . . . N] on a receiving unit i can be written as Δr i ∝−μr i +rjH(θ i −θ j ) and Δθ i ∝ω i +r j Q(θ i −θ j ), where the functions Q and H depend on the nature of the input, i.e. feedforward, feedback or inhibitory. The rationale for these equations is the following: (a) the effect of feedforward inputs on the amplitude is stronger for synchronized units; (b) excitatory feedforward and feedback connections are such that units that are simultaneously active tend towards phase synchrony; and (c) inhibitory connections tend towards de-synchronization; at the same time, they have a stronger depressing effect on the amplitude of synchronized units, and correspondingly a weaker effect for de-synchronized units.  
         [0030]     Formally, the update equations for the units in the input and output layers are:
 
{dot over (θ)} 1     i   =ω 1     i   +Σ j   W   i   B   j ρ 2     j   τ 1     i   Φ B (θ 1     i   −θ 2     j   )  (1)
 
{dot over (τ)} 2     i   =−μτ 2     i   +Σ j   W   i   F   j τ 1     j   Γ Fr (θ 2     i   −θ 1     j   )−γ τ Σ k   G   ik ρ 2     k   Γ 1     r   (θ 2     i   −θ 2     k   )  (2)
 
{dot over (θ)} 2     i   =ω 2     i   +Σ j   W   i   F   j τ 1     j   Φ F     θ   (θ 2     i   −θ 1     j   )−γ θ Σ k   G   ik τ 2     k   Φ I   θ (θ 2     i   −θ 2     k   )  (3)
 
 Φ F     θ   (φ)=sin(φ)Γ F     θ   (φ), Φ B     θ   (φ)=sin(φ)Γ Bθ (φ), Φ I     θ   (φ)=−sin(φ)Γ I     θ   (φ), Γ α =e −(1−cos(φ))           /2σ     α           where α={F τ , I τ , F θ , I θ }; the initial conditions for the presentation of an input (t=0 is input onset) are:
 
θ 1     i   ( t= 0)=0∀ i, θ   2     i   ( t= 0)=0∀ i, τ   1i ( t )=χ i   (n)   ∀t≧ 0 τ 1i ( t )=χ i   (n)   ∀t≧ 0
 
 Finally, the upper layer&#39;s amplitude is rectified, such that {dot over (τ)} i ≧0∀i if τ i =0. 
 
         [0031]     During the learning stage, feed-forward, feedback and inhibitory connections are subject to plastic changes. These changes are a generalization of the simple Hebbian rule of synaptic update, based on the coincidence of activity between the pre-synaptic and post-synaptic units, i.e. the incoming and receiving units defined by a connection. The rules are written as follows:
 
 {dot over (W)}   i   f   j =β f τ i τ j Γ f (θ j −θ i )
 
 where f={F, B, I}. This implies that learning for the three classes of connections has the same functional form, and consequently the strength of inhibitory connections will increase if both units tend to be coactive. During the performance stage, the response equations are identical, the only difference being that learning is turned off. Further changes may be implemented either gradually over the course of learning, or discretely at the transition between the learning and performance stages. For the results presented in the next section, learning constants were decreased to approach zero with an exponential schedule. 
 
         [0032]     Thus, during the learning phase, the learning network  100  creates a weight for each connection between the units. The weight of a connection can be changed according to a product of the amplitudes of the units connected. For example if the inputs A and B are received at the input layer, then the weight can be changed as the product of their amplitudes. In addition, the network  100  can also update in proportion to the phase difference between the two units connected.  
         [0033]     In the operational phase, for example a signal is received that has two objects (e.g., images). The network  100  produces an output that can recognize that the input contains an image (e.g., a face). The network  100  can also recognize the presence of a mixture of images (e.g., a face and a car). The network  100  was trained with pictures of faces and pictures of cars. The network  100  can also segment which pictures came form the face and which came form the car.  
         [0000]     Dynamical Segmentation.  
         [0034]      FIG. 2A  shows an instance of the input ensemble used to test the segmentation algorithm.  FIG. 2B  shows the behavior of the network after learning. The traces (a) and (c) show the amplitude and phase response upon presentation of an input from the training ensemble. In traces (b) and (d) we see the response to the presentation of a mixture. For the amplitude, the evolution is shown since input onset; for the phase, only the behavior after convergence is shown. Empty circles correspond to the traces from the input layer and circles with a dot inside represent the units from the input layer.  
         [0035]     The algorithm was run on training inputs drawn uniformly for ten-dimensional vectors. Time is in simulation steps. The units compete to represent the input, until one wins and shuts down the others. This leads to a global synchronization at small phase difference of all lower layer units with the winner, which emerges after 3-4 cycles as determined by the mean frequency of the oscillators. The existence and stability of 1:1 synchronized states is predicated upon a relatively small spread of natural differences. In contrast, when a mixture of training examples is presented there is also synchronization (emerging in a similar time scale), but competition leads to the emergence of two winners, which divide up the lower layer in terms of phase difference.  
         [0036]     Our initial results show that indeed the system is able to separate or deconvolve a mixture of two components, drawn at random from the training ensemble, into its original values. In short, we found that over 1,000 different realizations of the input ensemble, 75% of the cases we correctly deconvolve. The other 25% consisted of wrong winners emerging, including 1% of cases when at least one of the components was correctly identified. More importantly for the goal of the this paper, of the 75% correctly deconvolved cases, there was 93.6% accuracy for segmentation.  
         [0037]     For the deconvolution, the identity of the input is based on the elements of the upper layer whose amplitude exceeds a threshold, r i ≧0.1. After learning, presentation of a pure exemplar always leads to a single winner. On the other hand, the segmentation is computed by assigning to each winner in the upper layer the units in the lower layer whose phase is closest, after settling. More precisely, let unit i in the upper layer be the winner for an input x 1 , and let unit j be the winner for input x 2 . Suppose units i and j in the upper layer are the winners for a presentation consisting of a mixture of two inputs, x 1  and x 2 , indicating that deconvolution has taken place correctly. Let the phases of units i and j in the upper layer be θ 2i  and θ 2j  respectively. Consider a unit k in the lower layer with phase θ 1k . The behavior of the network is such that the phase of the k th  unit is usually synchronized with the phase of one of the winners in the upper layer. Suppose, without loss of generality, that θ 1k ˜θ 2i , i.e. the phase of unit k in the lower is close to the phase of unit i in the upper layer. We observe another interesting behavior in the network, in that x 1k &gt;x 2k , i.e. the input at location k is higher for the first input. In other words, the network is able to implicitly determine which input is higher at a given position, forming the basis of segmentation. We say that the input at location k is correctly segmented if the following holds true: let the higher of the two inputs at location k be x 1 , and let the winner at the upper layer that responds to input x 1  be i. If the phase of the input unit at k, θ 1k ˜θ 2i , then input at location k is correctly segmented. The overall segmentation accuracy for a given pair of inputs is determined by counting the number of units in the lower layer that are correctly synchronized with the appropriate winners in the upper layer.  
         [0038]     FIGS.  3 A-C show a concrete example to illustrate segmentation.  FIG. 3A  shows two inputs used.  FIG. 3B  show mixture of the two inputs is presented to the system.  FIG. 3C  shows the winner for input  1 , W 1 , and the winner for input  2 , W 2 . The system&#39;s phase response to a mixture of signals is shown in  FIG. 3C , where it can be seen that the segregation of phases and the implicit rule that if x 1j &gt;x 2j  then the input element at location j follows the phase of the winner for input  1 . Thus, the phase at each input unit shows which input is higher at that particular point.  
         [0039]     We investigated the relationship between the determinant of the input matrix and the error rate for deconvolution. Let d be the determinant of the input matrix. This was converted to a normalized form that D is equal to the tenth root of d, as the dimensionality of the input matrix was 10, and the input vectors are normalized to unity.  
         [0040]      FIG. 4A  is a conditional probability distribution for deconvolution failures and summarizes the results of over 500 trials.  FIG. 4B  shows the average segmentation accuracy versus the dot product of the inputs selected for mixing.  
         [0041]     We compute the conditional probability for deconvolution failure p(F\D) as a function of D (the unconditional distribution of D is approximately Gaussian—not shown). Ignoring the noise at the tail ends of the distribution, we see that the failure to deconvolve is not dependent on the determinant of the input matrix, indicating that the method is quite robust. If there was a dependence, we would have expected that the failure to deconvolve would increase as D decreased. However, this does not appear to be the case.  
         [0042]     We further characterized segmentation by measuring the relationship between the dot product of the inputs selected for mixing, and the segmentation accuracy after deconvolution. One expects that as the inputs become more similar, i.e. as their dot product increases, the segmentation task becomes more difficult, and the segmentation accuracy will decline. A total of 351 cases were analyzed to produce this result. As the value of the dot product increases, the segmentation accuracy decreases from 100% to about 80%.  
         [0043]     Average classification accuracy versus the noise level a normalized form, as the dimensionality of the input matrix was 10, and the input vectors are normalized to unity.  
         [0044]      FIGS. 5A-5B  show the effect of adding noise on the deconvolution and classification performance.  FIG. 5A  shows the average deconvolution accuracy versus the noise level.  FIG. 5B  shows the average classification accuracy versus the noise level. We added uniform noise up to a maximum noise level, η, where η was varied between 0.05 and 0.7. Thus the input was perturbed by noise drawn from a uniform distribution between [−η, η]. The resulting input was remapped to be positive, and normalized. As can be seen in FIGS.  4 A-B, the performance of the system is robust with respect to noise. Furthermore, the classification accuracy deteriorates less rapidly than the deconvolution accuracy.  
         [0000]     Entrainment Analysis.  
         [0045]     The conditions for entrainment of limit cycle oscillators have been studied at length, beginning with the pioneering work of Winfree and Kuramoto. A. Winfree, “The geometry of biological time,” New York: Springer-Verlag (1980); Y. Kuramoto, “Chemical oscillations, waves, and turbulence,” Berlin: Springer-Verlag (1984). However, the class of oscillators that we discuss here have been less studied. In particular, Kuramoto demonstrated that the interaction between limit cycle oscillators with comparable natural frequency can be expressed as a sinusoidal function of their phase difference. Although a thorough investigation of the dynamical properties of the system introduced herein is beyond the scope discussed herein, we present here a simplified analysis of entrainment conditions, which shows qualitatively similar properties as those described in the case of pure relaxation oscillators. Specifically, we will use pure sinusoidal functions for the phase interaction terms, as opposed the ones used in our simulations.  
         [0046]     In the case of a full (upper layer) oscillator coupled to a reduced (lower layer) oscillator, or more generally an oscillator that receives only phase feedback. In this case, the update equations are:
 
{dot over (τ)} 2 =−τ 2 +τ 1   W   21  cos φ  (5)
 
{dot over (θ)} 2 =ω 2 +τ 1   W   21  sin θ  (6)
 
{dot over (θ)} 1 =ω 1 −τ 2   W   12  sin θ  (7)
 
 where φ=θ 2 −θ 1 . From the equilibrium conditions {dot over (τ)} 2 =0 and φ=0, the following equation can be derived for the phase difference under entrainment: τ 1 W 21  sin φ(1+τ 1 W 12  cos φ)=Δω, where Δω=ω 1 −ω 2 , This implies that if Δω&gt;τ 1 W 21 (1+w 12 τ 1 ) entrainment is possible. The condition for entrainment at small difference leads to: φ˜Δω/(τ 1 W 21 (1+W 12 τ 1 )), which makes evident that a driving or lower layer unit with high amplitude can synchronize with a small phase difference with the upper layer, and that this can also be achieved by a strong feedback connection. 
 
         [0047]     Now let&#39;s consider the case of two upper layer units that interact through mutual inhibition, receiving independent inputs. Simplifying, we assume that the lateral connections are identical and not too strong, so that the units do not shut each other down. In this case, we can write the equations as:
 
{dot over (τ)} 1 =−τ 1   +A−τ   2  cos φ  (8)
 
{dot over (θ)} 1 =ω 1 −τ 2  sin φ  (9)
 
{dot over (τ)} 2 =−τ 2   +B=τ   1  sin φ  (10)
 
{dot over (θ)} 2 =ω 2 +τ 1  sin φ  (11)
 
         [0048]     A similar analysis for the entrainment (or rather exclusion in this case) condition leads to: (A+B)(1−cos φ)=Δω sin φ. Clearly, φ=0 is a solution, but an unstable one. If (A+B)≅Δω, the solution is near π/2, more precisely φ≅π/2+Δω/(A+B)−1.  
         [0049]     Finally, we can analyze the behavior of this simple system when lateral connections are strong, which will be the case after learning. We write the amplitude update as:
 
{dot over (τ)} 1 =−τ 1   +A−W   12 τ 2  cos φ
 
{dot over (τ)} 2 =−τ 2   +B−W   21 τ 1  cos φ
 
         [0050]     For large connections strengths, one of the amplitudes will eventually reach zero, and therefore the steady state solution will say r 1 =A, r 2 =0. In this case, the entrainment condition is ω 1 =ω 2 −ω 12  A sin φ, and therefore the entrainment condition is satisfied by:
 
sin φ=Δω/( AW   12 ).
 
         [0051]     Therefore, while there has been described what is presently considered to be the preferred embodiment, it will be understood by those skilled in the art that other modifications can be made within the spirit of the invention.