Abstract:
The pulse-coupled automatic recognition system processes information utilizing parallel neural network and transmits information to and from the processing sites as time signals which encode in their pulse phase structure the geometrical content of the spatial distributions of light. A locating device then measures the coordinates of the spatial distributions (pulsating segment) and displays the coordinates of the pulsating segment on a screen for observation. The pulsating segment indicates the presence and location of the component of the input scene that corresponds to the segment.

Description:
The invention described herein may be manufactured, used and licensed by or for the Government for governmental purposes without the payment to me of any royalties thereon. 
     BACKGROUND OF THE INVENTION 
     Prior art in computers consists of analog and digital processors. The analog devices implement algebra and calculus functions for the purpose of solving systems of equations while digital devices implement arithmetical and logical statements in a serial fashion and do not use parallel processing. 
     The pulse-coupled neural network (PCNN), derived from the Eckhorn mathematical linking field model (see &#34;Feature Linking via Synchronization among Distributed Assemblies: Simulations of Results from Cat Visual Cortex,&#34; by R. Eckhorn, H. J. Reitboeck, M. Arndt and P. Dicke, Neural Computation 2, pp. 293-307 (1990)), is a higher order neural network which uses pulses and pulse coupling to form linking waves and time signals. It presents a general method of combining information, transmitting information to and from processing sites as time signals. The time signals encode in their pulse phase structure the geometrical content of the input spatial distributions and other feature information. It combines, or fuses, the information from separate sources by multiplying their time signals to form a composite time signal. The PCNN allows time-synchronous pulse activity as well as parallel processing through neural network algorithms. It contains both sums and products that can be combined to make logical rules for information fusion. 
     The pulse-coupled neural network model neuron has three parts: (1) a pulse generator (2) a linking mechanism and (3) receptive field structures. The network itself, as distinguished from the individual neuron, is composed of many such neurons all interconnected with each other and further connected to external signal sources and external systems. With reference to FIG. 1 which is a diagram of a model neuron, each part of the neuron is explained below in detail. In FIG. 1, as in the other figures, like numbers represent like parts and arrows denote the direction of signal travel. 
     (1) The Pulse Generator: 
     As shown, pulse generator 101 consists of threshold discriminator 103 followed by pulse former 105. The discriminator is triggered when an input signal U j , called the internal activity, exceeds the value of the threshold signal θ j  at some point in time. When such an event occurs, the discriminator sends a signal to the pulse former causing the latter to produce spike-like pulse trains. These pulses are produced at a fixed rate and are all identical. The pulse former output is the neuron&#39;s output, Y j , which feeds to other neurons and which also feeds back to threshold element 107. The threshold element is a leaky integrator, modeled by a first-order relaxation or exponential decay process. It is recharged by the pulses produced by the pulse former until it is driven above the level of the internal activity at which time the pulse generator stops making pulses. 
     (2) The Linking Mechanism: 
     Linking mechanism 117 allows an input signal, called the modulating linking input 1+β j  L j , to modulate the standard input, called the feeding input F j , to pulse generator 101. The modulating linking input is produced by adding, at summing junction 113, a bias offset to L j  which is derived from spatial and/or temporal groups of pulses from other neurons and algebraically multiplying the result at multiplier 111 by the feeding input F j  that comes from a standard video camera. The resulting product U j  is then used as the modified input to the pulse generator. The modulating linking input which travels by channel 109 is preprocessed prior to the biasing and multiplication that happen at summing junction 113 and multiplier 111, respectively. 
     (3) Receptive Field Structures: 
     The aforementioned pre-processing is done by receptive field structures 115 of the model neuron. The temporal preprocessing usually consists of convolving the input pulses Y(Kl) through Y(Kn) from other neurons (first to nth neurons) with exponential decay time kernels 119 through 121. These are first-order relaxation processes and are identical to that used in threshold element 107 except that the characteristic decay time of time kernels 119 through 121 is shorter than that of threshold element 107. Subsequent spatial pre-processing consists of forming a weighted sum, at summing junction 113 (which also adds the linking strength β), of pulses over a given spatial region. This weighted sum is the linking input β j  L j . 
     As illustrated in FIG. 1, the receptive field structures consist of means for receiving the pulses from other model neurons and a source of video image and combining them into appropriate input signals for the linking mechanism. Each model neuron can have many receptive fields, both feeding and linking, and the several receptive fields can each receive a pulse from the same transmitting neuron at the same time. The receptive field structures receive many inputs and generates only one output, U j . A receptive field can have a single or multiple connection points each of which is available for connection with one input pulse signal. A multiplicative weight (not illustrated) can be applied to the amplitude of the input pulse and the value of the weight can be different for every connection point. The connection point can further have a first-order relaxation process which acts on the weighted input pulse and produces an exponentially-weighted time average. All of the first-order relaxation processes can have different and distinct decay time constants and all of the receptive field connection points can have different time decay constants. Following the connection point weighting and the first-order decay processes, the signals are linearly summed at summing junction 113 and biased to become modulating linking input, 1+β j  L j . 
     SUMMARY OF THE INVENTION 
     Electro-optical pulse-coupled neural networks generate pulse patterns from optical images of cluttered input scenes. The pulse-coupled automatic object recognition system utilizes the pulse patterns to detect and recognize pre-selected objects in such cluttered environments. Time signals that are produced from these pulse patterns are characteristic of the various components of the input scene. The time signals that correspond to pre-selected objects are used selectively to open a time gate thereby transmitting light only when the pre-selected objects are present in the scene. 
    
    
     DESCRIPTION OF THE DRAWING 
     FIG. 1 is a diagram of a model neuron. 
     FIG. 2 shows a new electro-optical implementation of the neuron based on the principles of Eckhorn as explained above. 
     FIG. 3 illustrates a preferred embodiment of the automatic object recognition system utilizing the electro-optical implementation of the neuron as depicted in FIG. 2 above. 
    
    
     DESCRIPTION OF THE PREFERRED EMBODIMENT 
     Referring now to FIG. 2, the structure and operation of the electro-optical implementation and augmentation of the neuron based on the Eckhorn model is explained in detail. 
     Feeding optical input F j  originating from a standard video camera (not shown here) is input to first spatial light modulator (SLM) 201 and when linking optical input β j  L j  (from a source to be detailed later) passes through SLM 201, it multiplies with F j , thereby producing U j . U j  is then transmitted to be incident on reimaging lens 203 which, in turn, focusses the image onto first detector array 205. The first detector array, which may be a CCD array, converts the optical image of U j  into corresponding electronic video signal and inputs this electronic U j  to step function element 207 which performs the functions of the discriminator in the Eckhorn model neuron. If the electronic U j  exceeds the value of the threshold signal θ j  that is input to the step function element from second detector array 209, then the step function element goes from &#34;0&#34; to &#34;1&#34;. This action causes pulse former 211 which is coupled to the step function element to produce at least one pulse. However, usually multiple pulses are produced by the pulse former in spike-like pulse trains, one pulse per frame time. These pulses, thusly generated by the pulse former, are the pulse pattern output, Y j , of the jth neuron. Y j  is then simultaneously fed into second SLM 213 and a set 215 of other neurons surrounding or in the vicinity of the jth neuron. 
     The processing of Y j  that goes to second SLM 213 is discussed first: While the pulse pattern Y j  is visible on the second SLM, a light from first suitable source 217 is shone through the SLM, thereby reimaging Y j  onto phospher-coated glass plate 221 via re-imaging lens 219. Upon receiving a lit-up pulse, the phospher-coated glass plate brightens then subsequently undergoes a slow decay. When Y j  is a pulse train, what emerges from glass plate 221 is a pattern of pulses, all decaying in a staggered fashion. These decaying pulses are re-imaged by third re-imaging lens 223 onto second detector array 209, the video data stream into which is perfectly synchronized with the data stream from first detector array 205 to step function element 207. The second detector array, which may be a television camera, reads the decaying pulses and sends corresponding electronic video signals, θ j , to the step function element. θ j , received by the step function element from second detector array 209, is subtracted from U j  received from first detector array 205. If the result is &#34;0&#34; or negative, then nothing occurs at the step function element. However, if the subtraction yields a positive result (i.e. U j  &gt;θ j ), then the step function element goes from &#34;0&#34; to &#34;1&#34; at that instant and pulse former 211, that is coupled to the step function element, produces at least one pulse output Y j . The step function element is brought back down to &#34;0&#34; when the pulses produced by the pulse former while the step function element remains in &#34;1&#34; is enough to result in a new θ j  that is greater than U j . In such a case, the subtraction of θ j  from U j  would result in a negative sum thereby bringing the step function element from &#34;1&#34; to &#34;0&#34;. 
     As stated earlier, Y j  is also fed into other neurons (pixels) 215 which surround or are in the vicinity of the jth neuron, all of which neurons being the pixels of third SLM 225. When receptive field structures 115 of a given sub-set of neurons receive Y j , light from second suitable source 239 is incident on SLM 225 and transforms Y j  into an optical image which is then imaged onto first summing lens 229. The first summing lens is located between second phospher-coated glass plate 231 which itself is located at the re-imaging plane and third SLM 225 and is further positioned such that the focal point of the first summing lens determines the size and location of such sub-set of neurons on SLM 225. The first summing lens may be set at the focal plane or deliberately set out-of-focus. If it is set out-of-focus, then each point of the image Y j  on second glass plate 231 receives contribution from several points of the image on the other neurons 215 of third SLM 225. A multiplier mask 227 that may be positioned anywhere between third SLM 225 and second glass plate 231 multiplies the optical image of Y j  emerging from the third SLM with linking strength factor, β. If the mask is set arbitrarily out-of-focus, then each point of the mask will multiply by its particular value the several points of the image on the other neurons 215. Thus the mask&#39;s transmissivity at each point gives some type of weights. However, for the special case where the mask is located at the aperture of first summing lens 229, the mask will give the identically weighted receptive field for all the points in the image field of the lens. From second glass plate 231, the resultant image which is the weighted sum out of the receptive field structures (as depicted in FIG. 1) travels to beamsplitter 233 that provides a bias offset by guiding into the optical system a beam of a given constant-intensity from third suitable light source 237. The biased image, now referred to as 1+β j  L j , is then re-imaged by second summing lens 235 onto first SLM 201 where it multiplies with feeding optical input F j  to result in modied input U j . Thereafter, U j  is processed as described above. 
     In the electro-optical implementation of the pulse-coupled neuron as illustrated in FIG. 2, the source of optical feeding input F j  and third light source 237 for providing the bias are always on, independently of the other elements of the neural network. The neural network is self-contained and the pulse pattern Y, the combined output of neurons 215, that is imaged onto third SLM 225 is the final output of the pulse-coupled neural network (PCNN). 
     This final output Y of the PCNN can be used in a system for automatic recognition of selected objects in a cluttered environment. FIG. 3 shows a diagram of the preferred embodiment of such a system. 
     The pulse pattern Y A  on PCNN 200A modulates collimated light beam 301 passing therethrough from first light panel 323. The resulting optical image of Y A  impinges on first focussing lens 303 which focusses the image to a point on single-pixel detector 305 which, in turn, produces electronic output signals in response to the focussed input image. These electronic output signals are the reference time signals of the various components of pulse pattern Y A  and are coupled into time correlator 307 that is suitable for recording and storing the reference time signals and is further capable of transmitting to time gate 313 the time signals that correspond to pre-selected components of the entire pulse pattern Y A . 
     Contemporaneously with the production of the time signals, another collimated beam of light 309 from second light panel 325 impinges on PCNN 200B and is modulated by pulse pattern Y B  before being incident on second focussing lens 311. It is noted here that Y A  and Y B  are identical to each other and PCNN&#39;s 200A and 200B have the identical number and geometrical arrangement of pixels. Upon incidence of the image of Y B , the second focussing lens, then, focusses the image onto time gate 313 which opens and closes in phase with the time signal input thereto from time correlator 307, thereby selectively transmitting the focussed light onto collimating lens 315. Such a time gate can be single-pixel optical spatial light modulators. The collimating lens, then, re-collimates the light and transmits it to impinge on video image detector array 317, such as a CCD array, that has the identical number and geometrical arrangement of pixels as do the first and second PCNN&#39;s. The output of detector array 317 is further input to centroid locator 319 which measures the coordinates oft he light distribution on the detector array and displays the information on screen 321. Since time gate 313 opens only in response to the reference time signal input from time correlator 307 and the input reference time signals are those of a pre-selected component of the pulse pattern Y (corresponding to a pre-selected object in the original scene that is represented by the pulse pattern), the output light distribution on detector array 317 is primarily that of the pre-selected object image. Therefore, the coordinates produced by centroid locator 319 correspond to and are indicative of the location of that pre-selected object in the original scene and the displayed location of the pre-selected object bears the same relation to the screen as does the object itself to the entire scene. 
     Although a particular embodiment and form of this invention has been illustrated, it is apparent that various modifications and embodiments of the invention may be made by those skilled in the art without departing from the scope and spirit oft he foregoing disclosure. Accordingly, the scope of the invention should be limited only by the claims appended hereto.