A novel optical illusion based on visual-acoustical sensory interaction, possible
applications
R. Micheletto
Yokohama City University, Graduate School of Nanobioscience, Department of Nanosystem
Science, 22-2 Seto Kanazawa-ku, 236-0027 Yokohama, Japan
Abstract: We studied the influence of mechanical vibrotactile signals in the acoustic range to thevisual perception of flickering images. The "flickering" images are shown on a CRT screenintermittent at about 75 Hz, so in normal conditions are perceived as constant stable images.
However, if presented together with the controlled acoustical vibration described in this paper,
an illusion is perceived. The images appears to float out of the screen, while the rest of the room
is still perceived normally. This illusion can be employed to realize an apparatus and do intensive
investigation in order to find new way to study retinal diseases, do visual perception diagnostic
tests, and generally study optical perception of the single retinal cells on a live and conscious
human subject.
Introduction:
We know that images that reach our visual system (the retina) are elaborated partially in theretina and then in the brain. Finally these signals become a “high-level” sensation that we call“perception” of the object.
However, there is a great difference between what is the actual optical signal on the retina, andthe high-level perception we feel. For example, we may look at the traffic while we walk in thestreet. We perceive cars and people moving around. However, the actual signal in the retina is avery complex dynamical pattern of images, shifted in time and space because of saccadic
movements, vibration of the eyeball and other perturbations. How the visual system can put
order in this extremely complex pattern of stimuli is unknown (see for example: T. Fumihiko; M.
Ken “Spatio-temporal dynamics of the visual system revealed in binocular rivalry”
Neuroscience letters (2005), 381(1-2), 63-8).
We established a new laboratory called 知覚情報科学(Sensory Information Science) inYokohama City University. We conducted first original tests of visual perception via aninnovative double-stimuli setup. The method use an optical blinking pattern and an externaldelicate mechanical vibration of low frequency applied below the jaw of the subject.. With thismethod it was discovered that images presented on a screen are perceived with unstableoscillations due to the interaction
between the double stimuli.
It is difficult to describe a subjective
perception, however in the left figure
here we show the real patternpresented on the screen (screen isblinking at 70Hz) and we give on theright a representation of the perceived
pattern (illusion). We can alter thisillusion and making disappear adjusting the two frequency parameters f1(blinking) and f2(vibration). We consider this particular mechanical-optical stimulation as our discovery in thisfield and we are going to publish it as soon as we have solid data.
Illusion effect description:
As said above the images that come to the retina are not stable and disturbed by mechanicalperturbations and noises. Out brain in real time and without effort is able to filter the signal andcreate the perception of a clean and stable image. This "cleaning up" elaboration process thatthe brain does is a heavy and complex task. However, we are not aware of what is happening,
nor we are aware of the effort involved. It is a process that is unconscious and transparent to us.
Sure, it would be very nice to make it surface to the conscious level and study its
characteristics.
We succeeded to devise a simple experiment that does - somehow - that. The experiment
outlined here succeeds in interfering with the the visual elaboration process, producing “errors”
that can be perceived at the cognitive level. It deals with the interaction between acoustic
vibrotactile signals and the perception of flickering images.
To produce flickering images in
simple way, we used an old
cathodic tube monitor (CRT).
This once common device draws
images point by point with anelectronic brush that scan thescreen at a frequency of 75Hz.
Still images are shown, while asubject observe them quietlyseated on a chair. In normal
conditions these images appear
stable.

Fig. 2: The image presented tothe subjects on a CRT flickering
at 75 Hz.

Fig. 1: The very simple way weapplied the acoustical perturbation
Why it is so if the images areto the subject retina. The vibration
actually flickering at 75Hz ? The
is in the 50 – 100 Hz range of very
standard answer is: because the low intensity (almost inaudible)

persistence on the images on theretina is much longer than the scan speed. So the brain has enough time to “integrate” thepattern and create a stable complete image for us. The same thing happens when we go to thecinema. Different images are flickering very fast in front of us, yet we perceive a continuous
smooth movement for similar reasons.
We want to interfere with this process, so we devised the following method: a controlled
acoustical vibration of about same frequency as the display flickering is transferred to thesubject. To do that, we used a small wooden stick glued to a round plastic cylinder that is placed
horizontally under the chin as shown in figure (1). The stick is mechanically connected to aspeaker that the subject simply holds in his hand. Because of its low amplitude and frequency thevibration is barely audible, however it is actually transmitted to the whole head.
The frequency is generated by a sinusoidal function generator while the subject is seatedcomfortably looking at a geometric image on the screen. We tested several images and all ofthem produced the desired effect; in the experiments presented here we used the image in fig.
(2).
Results:
While varying the frequency and amplitude of the stimulus, we asked the subjects to say when
the perception was becoming unstable.
We found that when the audio signal reaches a frequency close to that of the screen flickering,something interesting happens. While the rest of the room appears unchanged to the subject, theimages presented on the CRT screen begin to move and appear unstable. The images displayed
are distorted with perturbations that change in strength and shape in line with the audio signal
intensity and frequency.
We asked the subjects to report 4 levels ofintensity to quantify this subjective
phenomenon: no illusion (image perfectlystable), weak, medium and maximum. In figure(3) are plotted the frequencies of maximum
effect for different subjects. The illusion
appears and decays with a Gaussian profile.

Fig. 3: The frequencies that resulted in the maximum
illusion strength for several subject. Some subjects
performed the experiments twice or three times, forthem the error bar is given.

It is worth repeating that all other objects in
the room are perceived as perfectly still, only
the images on the display appears tooscillate.

Proposed device:
A single photoreceptor is about 40 micrometer long of about 1 or 2 micron of diameter. A low
power intermittent beam of light is originated from a LED. Light will have an angular divergencecontrollable by a system of lens. The geometry of the eye implies that a 1x1 degree (deg) will
cover an area of about 300x300 square micrometer, the know density of cones in the retina is ofabout 20.000 receptors per square degree. Considering these conditions our specially designed
optical stimulus will have to cover an area of 0.01x0.01 degrees, that in this geometry
correspond to 1-2 square micron. This localization it is not forbidden by diffraction andrealizable by standard passive optical elements.

Our specially designed optical source will beconnected to a programmable electronic
circuits able to flash the light beam for adesired length of time and with a controllable
repetition rate (impulse duration andfrequency).
The subject will perceive this optical stimulus
Fig 4: The scheme of a proposed device to test optical on a single cone receptor cell and the doctors
can study the ability of the subject to do so in
cognition characteristics
function of the signal duration and frequency ina systematic way. Our specially designed box will include the secondary vibration actuator. This
soft rubber device will located on the window of the box on its lower side. In this way the subject
will put his jaw on it and will receive a very delicate, low frequency vibration signal on his skull.
This vibration is naturally transmitted to the eyeball that contains the retina.
The controllable mechanical perturbation will shift the retina of a few microns. This small
measure is enough to shift the optical signal on the retina on adjacent cells at regular interval oftimes. This perturbation will confuse the retinal elaboration system and the subject will perceive
an optical illusion: a sort of fluctuating optical source. The source will appear steady in intensity,
but chaotically shifting in its position in space. The adjustment of the two frequency involved(optical source repetition rate) and the vibrational oscillation will control the intensity anddominance of the effect. We verified empirically that the illusion disappears if this condition isrealized:
The nature of this optical illusion is the center of our exploratory research. The intrinsic
properties of the retinal cell photoreceptor as well as the elaboration properties of retinal
ganglion and the whole retina and the brain are responsible of the effect. With the aid ofprofessor N. Mizuki in the Yokohama City University ophthalmology group several systematic
calibration test will be performed to optimize the experimental setup
Conclusions:
We found a novel optical illusion related to the interaction of visual and vibrotactile stimulus, wedescribe here the illusion in details and give experimental quantitative results that describe itscharacteristics.
Also we propose a device that use this effect as a method to detect the functioning andperformance of subject visual system. This device is described and and also it is shown who it
can be employed to realize an apparatus and do intensive investigation in order to find new way
to study retinal diseases, do visual perception diagnostic tests, and generally study opticalperception of the single retinal cells on a live and conscious human subject.