From: jpj@jet.uk (jean-paul jeral)
Newsgroups: comp.ai.neural-nets
Subject: Re: Complex Numbers
Summary: Re: Complex Numbers
Keywords: neuroids complex
Date: 8 Apr 92 12:18:52 GMT
Organization: Joint European Torus


Hi N-N netland.
The posters about complex numbers neural nets kicked me in writing
what follows:

-----------------------------------------------------------------------
Complex numbers could be used to model not only the strenght
(firing rate) of the pulse train out of a neuron but also its
phase.

Apparently this is useless if we suppose that neurons outputs
are not correlated.

However if we take 2 outputs from the same neuron, afflict
them with a different propagation time, and
connect then as entries to a second neuron realizing, e.g.
the absolute value of the vector sum(a+b), its output would exhibit
maxima when the difference in propagation time is an integer
multiple of the period of the firing of the initial neuron.
If we assume that the first neuron is physically limited in its
rate of firing so that only the first maximum will be reached,
this would pintpoint a precise rate of firing, i.e.
mark a LEVEL of activity in the output of the first neuron.

I do not pretend that this correspond to any physical reality,
only to show that some functionnality could be implemented
with complex numbers. I also remembered confusely having read that
circuits with difference in propagation time where used (or
postulated?) in the auditory circuits of the brain for stereo
perception.

-Jean-Paul Jeral

------------------------------------------------------------------------
I mailed the previous text to Gavin Flower and here are his comments.
(Gavin writing):
------------------------------------------------------------------------

    (NB I make no claim to having thought these
        through in any detail)

One idea I (only just) thought of was:

What if only the "REAL" part of the sum of the weighted inputs were affected
by the output function and the "IMAGINARY" part to be transferred unchanged
(except, possibly, for being scaled)?  Allowing the weights to be "COMPLEX"
numbers.

Another idea (partially inspired by your comments), interpreting the summed
input as a complex number incorporating phase, like in AC circuit theory.
Using the REAL part now & the IMAGINARY part to be carried over to the next
input cycle.

-Gavin
------------------------------------------------------------------------

Now I (Jean-Paul) wish to add a few things to what I wrote previously:

As I did not consider that two neurons could have outputs correlated in
phase, I used a single neuron with two outputs in my previous example.
However, I found out that this correlation is quite possible, at least
for auditory neurons, as their firing rate has been shown to be
phase-locked to some degree to the phase of an incoming sinusoidal
sound wave (but the neuron was not firing for each sinusoid)(1). So if we
consider neurons firings on left and right side of the head, as both
are locked to the same external signal, their firings will be correlated.

Therefore it was not necessary to imagine outputs from the same neuron.
However let us remark that for the original configuration, if we can
assume the transfer function of the first neuron is the usual sigmoid,
the transfer function of the whole system (first + second) becomes a
bell-shaped curve.

I do not think I will be able to find the reference to the stereo
processing I mentioned at the end of my first section.

About the comments by Gavin: it seems that he is concerned by
simulation (mention of an input cycle) while I was writing
about a modelization - trying to show that if real neurons
(or articicial pulsed networks) were realizing these operations beween
pulses trains that we can model with complex numbers, the results
migth be useful (and therefore wet neurons with these features
might possibly exist).
Anyway, in Gavin's first suggestion I do not see clearly
if the numbers he uses are complex numbers (i.e. have to
behave in all operations as such) or could be simply a couple
of reals.

(1) In E. F. Evans, Functionnal anatomy of the auditory system, in
    H. B. Barlow and J. B. Mollon (Eds) The senses, Cambridge University
    Press, Cambridge, 1982, 284-286.

-----------------------------------------------------------------------

Comments welcome.
- Jean-Paul Jeral   (jpj@jet.uk)
(the unimaginative disclaimer that follows is added *automatically* :-)
- Disclaimer: Please note that the above is a personal view and should not 
  be construed as an official comment from the JET project.
Xref: sparky comp.ai.neural-nets:1901 bionet.neuroscience:202 bionet.info-theory:417