Forehead Electro-tactile Display for Vision Substitution
Hiroyuki Kajimoto1

Yo n e z o K a n n o 2

S u s u m u Ta c h i 3

T h e U n i v e r s i t y o f To k y o

EyePlusPlus, Inc.

T h e U n i v e r s i t y o f To k y o

ABSTRACTThe Forehead Retina System-composed of a small camera and
512 electrodes on the forehead-captures the view in front,
extracts outlines from the view, and converts the outlines totactile sensation by electrical stimulation. Forehead skin waschosen for stimulation in consideration of usability. An
electrotactile display was used so that the system becomes small
and durable. A gel layer was designed to prevent an unpleasantsensation during stimulation. The system primarily aims toenable the visually impaired to "see" the surrounding
environment..
K e y w o r d s : tactile display, electrocutaneous display, electricalstimulation, TVSS, visual prosthesis, virtual reality
1

INTRODUCTION

According to a WHO report in 2003, up to 45 million people arecompletely blind and 135 million people have low vision [11].
However, there exists no standard vision substitution system fortheir use in their daily lives. The goal of our project is to provide
a cheap, lightweight, yet fully functional system that provides
rich and dynamic 2D information to the blind.
The Forehead Retina System (FRS)—composed of a small
camera and 512 electrodes on the forehead—captures the viewin front, extracts outlines from the view, and converts theoutlines to tactile sensation by electrical stimulation (Fig. 1).
Using this device, the users can “see” the surrounding
environment with their forehead skin, without using their eyes.
Although the system is a combination technology ofcomputer vision, tactile display, and wearable interface, thispaper mainly discusses the display aspect of the system because
forehead electrical stimulation seems quite unusual.
1.1

Related works

There have been numerous proposals to restore the sense ofsight by surgical means [1][2]. Direct stimulation of the visualcortex by an electrode matrix accomplished by Dobelle [1] is awell-known example. However, these technologies are not yet
mature and may require at least a decade before they are put topractical use.
Further, there have been many studies on sensory
substitution, in which visual information is converted to tactile
or aural information. Meijer [10] proposed the vision to aural
conversion system. His idea involved expressing 2D informationby combining the frequency domain information (spectral
distribution) of sound and the position information of the soundsource using binaural earphones.
However, the system requires long-term training, and it
supposedly hinders sound from the environment; sound iscritically important for the visually impaired to grasp thesurrounding situation.
1
2
3

e-mail: kaji@star.t.u-tokyo.ac.jp
e-mail: kanno@eyeplus2.com
e-mail: tachi@star.t.u-tokyo.ac.jp

Fig. 1 Forehead Retina System (FRS).

On the other hand, the vision to tactile conversion system
was first developed by Collins [3], who used 400 vibration
motors on the skin of the back and a CCD (Charge CoupledDevice) camera. He called the system a tactile vision
substitution system (TVSS).
However, although many similar systems have been
proposed since then, they have not become widely used for thefollowing two reasons:
One is the technical aspect of the stimulation. For tactile
devices using a large number of mechanical actuators, thesystem becomes heavy, expensive, and requires a considerable
amount of power.
To make the device small and energy efficient, we must
vigorously utilize mechanical resonance so that input energy is
not rapidly converted to heat. Many works chose 200 to 300
Hz as the resonant frequency, which is the most sensitive
range for human tactile sensation. However, it is also thefrequency that is clearly audible to the user. Therefore, thenoise becomes a considerable problem if a few hundred
actuators are required to be used simultaneously.
The other reason pertains to the region of stimulation.
Many previous proposals placed vibration motors on thewaistcoat to stimulate the skin on the back. Although awaistcoat is definitely “wearable,” its use is not as effortless as
the most successful carry-on devices such as eyeglasses andcell phones.
Many other works used the fingertip as a stimulation point
because the fingertip skin has the highest spatial resolution inhuman body. However, because our purpose is vision
substitution rather than conventional tactile display, we cannot
choose the finger as a stimulation point because it is used forother purposes in many situations.

1.2Electrical stimulation

Electrical stimulation is lightweight, cheap, scalable, andconsumes much less energy. At the same time, it does not
generate sound and is free from mechanical resonance.
Therefore, many works used electrical stimulation for tactile
display [9]. Note that Collins [3] has also explored electricalstimulation in his TVSS.
Kaczmarek and Tyler [10] have developed a 144-pixel arrayfor the tongue. Electrical stimulation on the tongue is reasonablebecause it is always wet and has a low electrical resistance sothat tactile sensation is generated with low electrical voltage.
The tongue is also the location that has the second highest tactile
resolution. However, putting something inside one’s mouth
seems uncomfortable and the system would require
sophisticated design.
We propose to use the forehead as a stimulation point.
Although electrical stimulation has a long history, stimulation offorehead skin is not a common approach. However, in reality, it
is quite reasonable because of the ease in putting the FRS on andtaking it off. At the same time, although the forehead skin has arelatively low resolution compared to the finger or the tongue, it
has a higher resolution than the skin on the back [13].
Furthermore, if the stimulation is applied on the back, finger, or
tongue, the coordinate system transformation to change theimage on the skin to that on an absolute coordinate system iscomplicated. However, if we use the forehead, thistransformation becomes unnecessary.
There were some works on forehead stimulation using
mechanical stimulation, especially for the navigation of theblind people [4][12]. However, as the system on forehead should
not be large or heavy, system with large number of actuators was
not practical.
2

SYSTEM

Fig. 2 shows the diagram of the proposed system called theForehead Retina System (FRS) and Fig. 3 shows the appearanceof the system during use. A CCD camera attached on a pair ofsunglasses (Ajoka Corp., SunGlasses Camera) captures the viewin front. A laptop PC (Pentium-M, 1.1 GHz) extracts the edges
and converts it to a tactile stimulation pattern. The pattern istransmitted to the driver circuit via a standard serial port (128kbps). 512 electrodes are driven sequentially to create the tactile
pattern. The entire process is triggered by the image capture
event, which occurs every 33 ms (30 fps).

Fig. 3 Appearance of the FRS during use.
Table. 1 Hardware specifications of the FRS.

CameraView
Angle
FormatRefresh
Rate
Lens

2.1

45 degElectrodes
NumberNTSC
Color
30 fpsDiameter

512(32 × 16)
2.0 mmInterval

3.0 mmPinholeWeight

100 g

Image processing

Fig. 4 shows how the image is converted to a tactile patternwhen a cup or a hand is presented.
Every 33 ms, an NTSC format image is captured. First, it
is converted to a 160 × 80 grayscale image. Second, theoutline edge is extracted by a Laplacian of Gaussian (LOG)
filter. Third, the image is scaled down to a resolution of 32 ×
16. By allocating 1 or 0 to each pixel according to thefollowing equation, we obtain a black and white binary image.

⎧1 (pixel value > threshold)⎨⎩0 (pixel value ≤ threshold)Fig. 4 The captured image is converted to a tactile pattern. (Top) Raw
image. (Middle) Edge extraction. (Bottom) Tactile pattern.
Fig. 2 Structure of the FRS.

The current system can only express a black and white
image, which is definitely not sufficient. In the near future, thesystem will be expanded to provide the tactile presentation of agrayscale image, which is achieved by converting the image
intensity to a stimulation pulse frequency.
2.2

Electrical stimulation

Although electrical stimulation has a long history, it was notapplied for welfare devices for two reasons. One is that stablestimulation was difficult. The other is that spatial resolution wasmuch worse than that of the mechanical stimulation.
However, in reality, these difficulties have already beensolved by previous studies. Stable control of electricalstimulation is possible by using very short pulses [3], and thespatial resolution can be the same as that of mechanicalstimulation by using anodic (positive current) stimulation, in
contrast to conventional cathodic stimulation [6] [10].
Therefore, it can definitely be said that the use of electricalstimulation as a tactile display is already a well-establishedtechnology.
However, there still exist two problems for the FRS. One isforehead stimulation; the other is numerous electrodes.
2.2.1

Forehead skin stimulation

There are two groups of sensory nerves in the skin. One is theA-β fiber, which has a diameter of about 5 μm, connected tomechanoreceptors. The other is A-γ and C fibers, which have adiameter of less than 1 μm; they generate unpleasant sensations
such as pain and heat.
In electrocutaneous stimulation, the electrical current fromthe surface electrode generates electrical potential inside theskin. The potential along the nerve generates membranepotential. If the potential becomes higher than the threshold, thenerve is activated and the associated sensation is generated.
If we stimulate the A-β fiber, the generated sensation issome type of mechanical sensation such as pressure andvibration; this is because the receptors that are connected at the
tip of the nerve are mechanoreceptors (Merkel cell, Meissnercorpuscle, Ruffini ending, and Pacinian corpuscle). The problem
occurs when A-γ or C fibers are also stimulated because theygenerate an unpleasant sensation.
Two parameters determine the electrical current threshold.
One is nerve depth, and the other is nerve diameter. The
electrical current threshold is inversely proportional to the nerve
depth or the square of the depth [18], while it is inversely
proportional to the nerve diameter or the square root of thediameter [18].
The significant difference between the finger and theforehead is the skin thickness. While the finger skin has a horny
layer that is more than 0.7 mm thick, the forehead skin has ahorny layer that is less than 0.02 mm thick. The layer has highelectrical impedance and can diffuse electrical current.
This is why the sensation elicited by fingertip stimulation is
not unpleasant. In the fingertip, all the nerves are beneath the
thick horny layer so that the distance between the nerves and thesurface electrodes is relatively constant. Therefore, the A-β
fibers for mechanoreceptors are stimulated while the other fibers
are not activated (Fig. 5 top left).
However, if the horny layer is thin, the variance of the nerve
depth greatly effects the stimulation. In many cases, thinner
nerves reside in the shallower part of the skin, resulting in an
unpleasant sensation such as pain and heat. Therefore, if we put
the electrodes directly on the forehead skin, a severe pain
sensation is elicited (Fig. 5 top right).

The general idea to prevent this problem is to put a
conductive layer between the electrode and the skin. A
hydrogel sheet is commonly used. The layer works as theequivalent of the horny layer so that the excitation of thethinner nerves by electrical current concentration is prohibited.
We may regard the layer as a spatial low-pass filter, which
diffuses electrical current.
There are four parameters for the design of the gel layer:
electrode distance, gel thickness, gel impedance, and skin
impedance. The relationships among these parameters are as
follows.
If the gel is thin, the electrical current is concentrated, andthus, unnecessary nerves are activated.
If the gel thickness is relatively larger than the electrodedistance, the electrical current mainly passes through the gellayer and cannot reach the skin (Fig. 5 bottom right).
Therefore, most of the electrical power is wasted in the gel,which is a severe problem for a wearable device.
Similarly, if the gel impedance is relatively lower than theskin impedance, the electrical current only passes through thegel.
On the contrary, if the gel impedance is relatively higher
than the skin, the electrical voltage drops mostly in the gel sothat a large part of the energy is changed to Joule heat in thegel layer.

Fig. 5 The fingertip skin is covered with a thick horny layer, which
contributes to stable sensation. Hydrogel layer works as the equivalent of
the horny layer.

In conventional electrical stimulation, the electrodedistance was quite large (typically about 3 to 10 mm);therefore, the above relationships 2 through 4 were negligible
because the current pathway inside the gel can be ignored.
Therefore, a thick gel with a fairly low impedance (10 to 50kΩcm) was used. It seems that the gel layer is regarded as aconvenient substitute for conductive paste, which stabilizes
contact.
In contrast, we used a much denser matrix of electrodes, inwhich the interval was 3 mm; diameter, 2 mm; and distance, 1
mm. Therefore, the optimal gel characteristics should bedesigned based on the four relationships.
By trial and error, we found out that the gel thickness
should be between 0.3 and 2.0 mm. The optimal thickness isaround 0.5 to 1.0 mm. The gel impedance should be muchhigher than that of the conventional conductive gel for electricalstimulation. It should be around 100 to 1000 kΩcm. The optimalvalue is around 400 kΩcm. If these values are satisfied,electrical stimulation without any unpleasant sensation isachieved; simultaneously, power wastage is minimized.
Note that the optimal thickness is about the same as theelectrode distance, and the optimal impedance of the gel and theimpedance of the horny layer are matched.
2.2.2

set to 50 μs. Therefore, it takes 25.6 ms to stimulate all theelectrodes. It is barely shorter than the cycle time (33 ms).
One primitive but crucial problem is that of the shape ofthe forehead. It is spherical rather than cylindrical, and it
varies from person to person. We divided 512 electrodes toeight units of 64 electrodes. Each unit is a square of about 30
mm. As the unit is small, it fits well on almost any adultperson (Fig. 8).

Numerous electrodes

Compared to the presently used portable electronic devices,most of the proposed “portable” welfare devices are not portablein reality. They are bulky, heavy, and have a short operationtime.
By adopting electrical stimulation, these problems have been
partly solved. Nevertheless, driving N = 512 electrodes withmore than 300 volts is quite a difficult task, and such a task
normally requires a large circuit space.
We used a high-voltage switching integrated circuit. Fig. 6
shows the application of our technique. Rather than preparing N
current sources, we used one current source and N switches
called half-bridges. The half-bridge is composed of a high-side
switch and a low-side switch. By switching, we selected ananodic electrode while all the other electrodes served as theground [14].

Fig. 7 Electrode board configuration. Each unit has 64 electrodes, shift
resistors and half-bridge switches [14].

Fig. 8 512 (32 × 16) electrodes module for the forehead.

3
3.1

Fig. 6 2D information is presented by high-speed switching [14].

The same technique has commonly been used in motor drive
circuits. This technique was previously proposed and utilized incortical brain stimulation [15] and the cochlear implant [16] forelectrical nerve stimulation. We utilized the same technique forelectrotactile display to achieve both a very large number ofstimulations and system portability.
We also used shift resistors so that the switch states are set
by serial communication (Fig. 7). In this manner, although wedrove 512 electrodes, we only used a few wires to connect thedriving circuit and electrode module.
Note that in this switching method, only one point isstimulated at a time. Therefore, the refresh rate is limited by the
number of stimulations. Currently the electrical pulse has been

EXPERIMENT
Two point discrimination

Two point discrimination test was conducted to determine
electro-tactile spatial resolution. Four points A-D wereselected as SS (standard stimulation) points (Fig. 9). After 30
pps (pulses per second) electrical pulses were applied fore 1.0s
and paused for 0.5s, the same pulses were applied to the otherpoint as CS (comparison stimulation). The CS point was 1 or 2
electrodes away from the SS point, which means -6, -3, 3 or 6
mm displacement. The displacement was either horizontal orvertical.
In the horizontal case, participants were asked to answer
whether the displacement was right or left (forced choice). 10
tests were conducted for each displacement at one SS point
and answer rates were obtained. The same experiment was
conducted for the vertical displacement case.
Fig. 10 shows the result of the experiment from one
participant. Horizontal axis is the displacement, and verticalaxis is the rate of the answer. In vertical displacement case,one electrode displacement (3 mm) was enough to answer the
displacement direction correctly. It suggested that 3 mm
interval electrodes are not unnecessarily dense for theforehead.

On the contrary, in horizontal displacement case, there arecertain offsets according to the SS position. This offset becomeslarger when the SS point becomes away from center. Thisphenomenon seems peculiar, because it means that although SS
and CS are applied to the same location, one feels certain
displacements. We are now trying to find out whether it is aparticular phenomenon of the electrical stimulation or generally
happens in mechanical stimulation.

frequency fluctuation, and change in sensation after long-timeuse due to sweat. These will be the topics of our next study.
The current system uses only basic image processing toconvert the visual image to tactile sensation. Further image
processing, including motion analysis, pattern and color
recognition, and depth perception is likely to become
necessary in the near future.
We must note that although many useful algorithms havebeen proposed in the field of computer vision, we cannot
always use them unconditionally. The processing must be
performed realtime (at most 33 ms or one frame latency), andthe system must be small and power efficient for portability.
Therefore, we believe that rather than incorporating elegant
but expensive algorithms, the combination of bare minimum
image processing and training is practical. We are beginning totest the system with the visually impaired to determine themost appropriate balance as good human interfaces havealways achieved.
REFERENCES
[1]
[2]

[3]
[4]
[5]
[6]

Fig. 9 Two point discrimination test setup.

[7]
[8]
[9]
[10]

[11]
[12]
[13]
Fig. 10 Result of two point discrimination test. Horizontal and verticaldisplacements showed different behaviors.

4

CONCLUSION

We proposed the use of an electrotactile display on the forehead
as a vision substitution system. Although the display has a large
number of electrodes, the entire system is lightweight andcompact. We tested the system and confirmed that problems
regarding sensation quality rarely occurred. Two point
discrimination test suggested that 3 mm interval electrodes are
not unnecessarily dense for the forehead.
The paper showed only the static capabilities of the display.
Other topics pertaining to the display include tactile recognition
using head motion (active touch), grayscale expression using

[14]
[15]

[16]
[17]
[18]
[19]

W. H. Dobelle, “Artificial vision for the blind by connecting a
television camera to the visual cortex,” ASAIO Journal, Vol. 46, No.
1, pp. 3–9, 2000.
M. S. Humayun, E. de Juan, G. Dagnelic, R. J. Greenberg, R. H.
Propst, D. H. Phillips, “Visual perception elicited by electricalstimulation of retina in blind humans,” Arch. Ophthalmol., Vol. 114,
pp. 40–46, Jan. 1996.
C. C. Collins, “Tactile television - mechanical and electrical image
projection,” IEEE Trans. Man-Machine Systems, Vol. MMS-11(1),pp.65–71, Mar 1970.
C. C. Collins, M. F. Deering, “A microcomputer based blind
mobility aid,” in Proc. of sixth annual conference of the IEEE
Engineering in Medicine and Biology Society, pp.52-55, 1984.
P. Bach-y-Rita, “Brain mechanisms in sensory substitution,”
Academic Press, New York, 1972.
H. Kajimoto, N. Kawakami, T. Maeda, S. Tachi, “Electro-tactile
display with force feedback,” World Multiconf. on Systemics,
Cybernetics and Informatics, Orlando, 2001.
H. Kajimoto, M. Inami, N. Kawakami, S. Tachi, "SmartTouch:electric skin to touch the untouchable," IEEE Computer Graphics &Applications Magazine, Vol. Jan-Feb, pp.36-43, 2004.
H. Kajimoto, S. Tachi, “FantaScan - an electric alternative for
Optacon,” in the first World Haptics Conference, Pisa, Italy, 2005.
K. A. Kaczmarek, “Electrotactile and vibrotactile displays forsensory substitution systems,” IEEE Trans. BME, Vol. 38, pp. 1–16,
Jan. 1991.
K. A. Kaczmarek, M. E. Tyler, “Effect of electrode geometry andintensity control method on comfort of electrotactile stimulation on
the tongue,” In Proc. of the ASME, Dynamic Systems and Control
Division, Vol. 2, pp. 1239–43, 2000.
P. B. L. Meijer, “An experimental system for auditory image
representations,” IEEE Trans. BME, Vol.39, pp. 112-21, 1993.
C. Wada and M. Asonuma, “A Proposal for a Display Method of a
Walking Support System for the Blind,” 8th ERCIM Workshop“User Interface For All,” 2004.
E. R. Kandel, J. H. Schwartz, T. M. Jessell, “Principles of Neural
Science,” Appleton & Lange, 2000.
World Health Organization, “Elimination of avoidable blindness,”
http://www.who.int/gb/ebwha/pdf_files/WHA56/ea5626.pdf
H. Takahashi, H. Kajimoto, N. Kawakami, S. Tachi, “Electro-tactile
display with localized high-speed switching,” 12th Int. Conf. on
Artificial Reality and Tele-Existence (ICAT2002), Tokyo, Japan,
2002.
K. E. Jones, R. A. Normann, “An advanced demultiplexing systemfor physiological stimulation,” IEEE Trans. BME, Vol.44, No.12,
pp. 1210-20, Dec. 1997.
H. McDermott, “An advanced multiple channel cochlear implant,”
IEEE Trans. BME, Vol.36, No.7, pp. 789-97, Jul. 1989.
D. R. McNeal, “Analysis of a model for excitation of myelinatednerve,” IEEE Trans. Biomed. Eng, Vol.BME-23, No.4, pp. 329-37,
Jul. 1976.
J. T. Rubinstein, “Analytical theory for extracellular electricalstimulation of nerve with focal electrodes, II passive myelinatedaxon,” Biophys.J., Vol.60, pp. 538-55, Sep. 1991.