CONTRIBUTION

Haptics as a Key Technology
in Man-Machine Interface

Professor Hannes Bleuler
Robotic Systems Laboratory (LSRO),
EPFL Switzerland

The haptic senses include many different modalities: sensing of force, texture, temperature, body posture etc.
While optical display screens and acoustic signals are standard interface modi for machine to man communication,
haptic interfaces are still at a research level. But many practical and commercial realizations are now emerging.
This article gives some examples from the biomedical robotics field, namely surgery telemanipulators, rehabilitation
devices and assist devices. But research results from haptic interfaces could benefit many other domains, including
the automotive.
Key Words: Haptic interface, Man-Machine Interface, Human-Machine Interface, Man-Machine Interaction

1. Brief Background on Haptics
Haptics, from Greek “äpsx” = “I touch”, is one of the“classical” five senses described by Aristoteles. Of these,
touch is the only one common to all living organisms.
While humans (or any other organism) can live without
sight, hearing (Helen Keller), taste or smell, they cannot
live without the sense of touch.
Of these five senses, touch (or “haptics”) is the mostcomplex one. It includes at least six modalities, for eachof which physiologists can identify separate receptor cells
or organs.
• Force sensing• Texture sensing• Proprioceptive sensing, i.e. perception of bodily joint
positions and muscle postures• Temperature sensing, combining temperature andheat sink capacity of the sensed object• The vestibular system (sensing of the direction of thegravity acceleration)• Pain sensing (nocioception)
A subtle combination of the first three modalities
allows the distinction of slippery or sticky.
A combination of the first four haptic senses allows thedistinction of wet/dry.
Reflecting on the complexity and richness of haptic
perception, it is surprising that these senses havesomehow often been underestimated in comparison tovision and hearing. It is true, that the sheer information
quantity of the visual sense seems overwhelming fromthe engineering point of view. This is partly due to the2fact that this information quantity can be technologicallyestimated with seemingly “good” accuracy, although
there is room for much debate also in this field. The
optical “refresh rate” is somewhere between 30 and 50
Hz, perfectly sufficient for smooth video streaming,with the corresponding number of pixels and colors. The
perception limits, it seems, can be defined with a certain
accuracy. Such considerations will give a number of bits/
sec.
Similar reflexions are even easier for acoustic
perception: A frequency range of about 16 Hz up to 20
kHz, a dynamic range of up to 100 dB, all this can again
be translated into bits/sec. The discussion on compression
formats and perceptual thresholds illustrates however the
shaky value of such estimations.
Now, to how many bits per second correspond the six
modalities of haptics listed above? Almost needless to saythat this is still an open question, estimates rely on a great
number of assumptions.

2. Haptics as Man-Machine Interface
A man-machine interface consists of input and output
devices. The words “input” and “output” obviouslyinterchange their meaning when seen from the humanor from the machine side. Most common input devices(from man to machine) are keyboards, buttons, joysticks, steering wheel, pedals. The most common output
device (from machine to man) is a visual display (watch,tachometer, lamp or LED, display screen). Add to thisacoustic devices, microphone as input and speaker

JTEKT ENGINEERING JOURNAL English Edition No. 1012E (2015)

−Haptics as a Key Technology in Man-Machine Interface−as output and we might cover well over 90% of manmachine interface. Thus we see a very different situation
for the two interaction directions
1) [human → machine]2) [machine → human]
While the first direction of interface, [human →machine] , is essentially haptic, the second one [machine→ human] is to a large extent optical or acoustic, and only
very indirectly haptic. (Of course we feel the vibrations
and accelerations in a car, but this is a secondary effect,
not essential for operating the car). The important
haptic interface of the driver and the car is through thecommand elements, the steering wheel, the pedals, the“feel” of handles, buttons, knobs. There we see a real andimportant haptic interface in action: The original functionof the steering wheel is clearly only a command function,
to give the direction of driving. Of great importance ishowever the secondary function, i.e. the haptic feedback
the driver gets from the steering wheel. This is essentiallya force feedback, including some vibrational content (isthe engine running or not? At what speed?). Or imagine
having to brake with a kind of joystick with no haptic
feedback, where braking torque is only proportional toposition and not to force on the pedal. It would seem quite
awkward, and we would have to rely all the more on thehaptic feedback signal of deceleration the car is giving us.
However, with some practice, one can become proficientat operation on visual feedback only: Think e.g. of videogame addicts.
These few examples have introduced the importanceof haptic interface in an every-day setting such as
driving an automobile. Haptic interface have actually
been a somewhat isolated topic of research for many1)
years (see e.g. Hayward ), but recently the number ofconferences (World Haptics, Euro Haptics, Asia Haptics)
and publications on the topic is rapidly increasing. I will
now come back to some examples outside the automotive
domain.

3. Application Examples in Biomedical Technology
3. 1 Surgery Robotics
Surgery has developed from open surgery to miniinvasive surgery and, since about 10 years, to robotic
surgery. This development is very logical, as the surgery
tools have become more and more sophisticated, inline with technical progress. Two essential pioneeringdevelopments in surgery have come from outside the fieldof surgery proper:2)
1) Anaesthesia, pioneered by Hanaoka Seishu in late Edothperiod (early 19 century) Japan andth2) Asepsis, discovered mid 19 century by Semmelweiss
in Vienna.
th
The 20 century has then seen the development of

JTEKT ENGINEERING JOURNAL English Edition No. 1012E (2015)

endoscopy introduced on a large scale by Karl Storz inGermany and Olympus in Japan. Endoscopy (seeing
inside the body through rigid or soft tubes) has quickly
led to the techniques of mini-invasive surgery, for as
doctors could see, they also wanted to operate throughlittle holes. This is why the word “endoscopy” thenacquired the meaning of not only seeing, but also
operating through small orifices, natural orifices aswell as small perforations. Needless to say that suchmini-invasive techniques, while very beneficial to thepatient (less trauma, less scars, less infection, shorter
hospitalization), are much more demanding for complex
surgical manipulation. Simple gestures such as suturingneed intensive training when done endoscopically.
Instrument makers now succeed in building endoscopes
which include a micro-camera and two tweezers of sevendegrees of freedom each. The total number of degrees of
freedom of such an endoscope (two tweezers, camera,
endoscope position) then easily exceeds twenty. Such an
instrument needs two to three surgeons to operate.
To the engineer, the solution out of this dilemma is
obvious: Motorize the instrument and build a commandconsole for the surgeon. The surgery instrument thus
becomes a telemanipulator (sometimes also called“master-slave system”), similar to the devices designed
60 or so years ago for the handling of dangerous (e.g.
radioactive) substances. Such a teleoperated surgery
system is called, somewhat abusively, surgery robot. It isof course not falling under the usual definition of “robot”
as autonomous (or automatic) machine, strictly speaking
it is just a telemanipulator. But it has the huge advantagethat the surgeon can again operate with his two hands in anatural way, not having to turn 20 knobs.
This situation explains the success of the DA VINCI
surgery robot from Intuitive Surgical inc. (Fig. 1) in thelast 10 years. So essentially the surgeon has regained
the mastery of his tools as he had it in open surgery,but now in a mini-invasive (“endoscopic”) mode, a bit
as if he would be “inside” the patient with his hands.
The cognitive neuroscientist would say, the surgeon“embodies” his tool in an efficient way, i.e. the motor
cortex of the surgeon’s brain identifies the tool as partof the surgeon’s body. This will be discussed briefly insection 3. 4.
However today’s surgery robots still have absolutely noforce feedback, they rely only on visual feedback throughthe endoscope camera, often only 2-D, sometimes 3-D
vision. This sets clearly the agenda for research insurgery robotics: We as engineers will need to develop
a true haptic interface, i.e. measure the forces at the tipof the operating tools (with sterilizable and if possible
disposable force sensors) on the “slave” side of thetelemanipulator and a kind haptic joystick (or handle)
on the “master” side (the command console), with a true3

−Haptics as a Key Technology in Man-Machine Interface−force-feedback. This will give back to the surgeon thefull sensation of working with his hands inside the body,especially if such an interface can be made multimodal(adding temperature, pulsation etc.)

Fig. 1 DA VINCI surgery “robot” (or telemanipulator) from
Intuitive Surgery inc
Up to now, such commercial telemanipulators have noforce feedback, they rely only on visual navigation by
the surgeon, seated at a console in the operating room,
nearby the patient. Such devices have become very
popular among surgeons and patients as well, as they
allow a very intuitive manipulation of mini-invasive
surgery tools.

3. 2 Rehabilitation
Another important domain of biomedical robotics where
haptic human-robot interface is at the very core is thefield of rehabilitation robotics. With the rapidly increasing
aging of the world population (especially of course inJapan and Western Europe), the need for professional
physiotherapy is growing much faster than the supply
of able therapists. It is established that stroke-patientsand patients from orthopaedic surgery (hip replacement)
need intensive and permanent physiotherapy in orderto regain (or not to loose) their physical abilities. Also
spinal cord injury patients (over the whole range of spinal
cord injuries) are in great need of daily physiotherapy.
The effectiveness of robotic tools for rehabilitation has
been clinically proven in many research publications,the number of such studies is rapidly increasing. But
the cost of actual devices is still much too high and thecomplexity of operating such tools is at the present toohigh to release them for unsupervised training, ideallyat home. Such devices, whether for the hand, the arm or
the leg are complicated to fit to the patient, so the muchneeded assistance to the physiotherapist is only slowly
progressing. Moreover, as therapy is so important to regain
motricity after an accident (vascular, nervous or other),
it is also essential to come up with device motivating thepatient to use them regularly. We show here an example ofa 2-dof rehabilitation device for the hand developed at ourlab and interfacing with a game in virtual reality (VR) on
the “slave” side of the manipulator. (Fig. 2)

4

Fig. 2 Two degree-of-freedom (d.o.f) hand rehabilitation
device designed at EPFL Robotic Systems Lab forhemiplegic children
As with most rehabiltation devices, besides anergonomically correct kinematics, motivation of thepatient is a key feature. In this case, the task consists ofmaneuvering an object through a labirynth. Kinematics
and multi-d.o.f.force feedback are adjustable over awide range.

3. 3 Assist Devices
Exoskeletons as human-assist devices appear inscience fiction dreams since quite some time. In the last
decade, several such systems are appearing, many ofthem in Japan. While Honda’s walk-assist system (Fig. 3)
seems to be targeted, among others, for factory workers,
Cyberdyne’s HAL (Fig. 4) is promoted as both, assistancefor elderly and “human enhancement”, i.e. increasing thephysical force of a human e.g. to carry heavy loads. This
last function is of course closely embedded in biomedical
robotics as an assist device for the daily care of elderly
and persons of reduced mobility, e.g. for a nurse carrying
a disabled patient.
While there are several encouraging pioneering devices
and research projects in this domain, carried out at many
institutions, practical commercially available devices ataffordable cost have yet to be developed. One of the main
challenges of such devices is the control. The exoskeleton“Rex” from New Zealand, designed for patients withcomplete lower limb paraplegy, is statically stable andis controlled by joysticks. It stabilizes the patient in an
upright position, but it is extremely slow (about 6m per
minute). The Cyberdyne HAL is controlled by EMG,
i.e. electrical signals picked up from the muscles of thecorresponding limbs. This seems to work reasonably wellfor healthy patients, it could also be a hopeful approachfor some partially disabled patients. The research question
is about how much autonomy the system should have toreally assist the wearer in his/her intention to move andwere to take the signals from. Here we drift away fromthe topic of haptic interfaces, this is a question of BrainMachine Interface (BMI), another rapidly expanding
research topic. It will not be investigated further in thisarticle.

JTEKT ENGINEERING JOURNAL English Edition No. 1012E (2015)

−Haptics as a Key Technology in Man-Machine Interface−Fig. 3 Walk assist device from Honda

not hinder normal walking. Technically speaking, thedevice should be back drivable, at least in the normal
walking mode. We are currently experimenting withdesigns allowing the motor to run always in the same
direction, thus avoiding acceleration losses. The core isa system with light weight brakes and clutches to reachthe oscillating movement of the leg. In designing sucha system, the complex joint kinematics of the humanskeleton must of course be thoroughly understood andaccommodated.

3. 4 Research in Cognitive Neurosciences

Fig. 4 “Robot Suit” human assist device HAL fromCyberdyne, Japan
It is controlled through EMG signals, i.e.
electrical potential signals taken from the muscles(electromyogram).

(a)

(b)

Fig. 5 Assist device for the hip joint designed at EPFL
Robotics Systems Lab
This device is intended for elderly patients, e.g. for remobilization after a hip joint operation or for stroke
recovery. The kinematic design aims at providing hightorque for the sitting-to upright transition and for stair
climbing while being as “transparent” as possible fornormal walking.

In Fig. 5, an assist device for the hip joint of elderlyis shown. The mechanical design of the kinematics issuch that there is high torque for the sitting to standingtransition and on stairs, while, for normal walking, thedevice should be as “transparent” as possible. In thiscontext “transparent” means that the device should

JTEKT ENGINEERING JOURNAL English Edition No. 1012E (2015)

Cognitive neurosciences have already been evoked
in section 3.1 on surgery robotics, when referring to“tool-embodiment” of a surgery tool by the surgeon.
This is just an example of a topic of great interest inneuroscience. “Embodiment” is the mechanism by which
the sensori-motor system (and our brain in general)
integrates our limbs, especially our hands, arms, legs andfeet as part of our body. The amazingly efficient operationof the complex multi-dof control of the human body relieson this “embodiment” mechanism, which could, from theengineering point of view (and with much simplification)
be described as the constitution of a dynamic model ofthe body in our brain, especially (but not only) in thesensori-motor cortex. Moreover, it is well known from
intensive research by cognitive neuroscientists that theskilful appropriation of a tool (fork, chopstick, cutter,screw-driver, surgical tool, music instrument, tennis
racket, bicycle, car etc.) relies exactly on this embodimentmechanism. Roger Federer “embodies” his tennis racket,
for his brain it has become part of his body. It is morethan just a metaphor to say that the violin has become “partof the body” of a great artist, it is true in a scientific sensethat his brain considers his music instrument as part of his
body.
From this description, it is only a small leap to the ideaof studying mechanisms of embodiment with the help ofrobotic (or mechatronic) tools such as haptic interfaces,
joysticks, telemanipulators etc. opens a wide new fieldof research. With a telemanipulator, we can operate ina virtual environment (Virtual Reality, VR) just as wellas in a real environment. For experimental work, we
thus have the huge advantage to have to realize only
the master side in hardware, the experimental set-up to
be manipulated can all be in software, in VR. Figure 6
shows such an experimental setup, where the mechanismsof embodiment can be studied in a controlled environment
and in complex scenarii.

5

−Haptics as a Key Technology in Man-Machine Interface−

5. Conclusions

Fig. 6 Telemanipulator acting in a Virtual Reality (VR)environment
The haptic master console designed by Dr. Marc
Vollenweider gives three-component force feedback
to two hand-held tweezers over a tetrahedral cablearrangement. In this way, there is a large workspace,large force feedback capacity, but no inertia to the
manipulator, as the heavy motors are in the fixed base.
The tweezer’s orientations are measured, but there is
no torque feedback. Realistic force feedback is possible
in this way. This device has been developed as a trainer
for surgical robotics and is now used for experiments
in cognitive neuroscience, but it has a large potential
for applications in other fields.

4. Other Application Examples (including
automotive)
After all that has been said, the importance of haptics
for the man-machine interface in the automotive
application field seems obvious. Especially the link tocognitive neurosciences provides a promising approachfor future research activity: indeed, driving a car, getting
to know it and “feel” at ease in it are all indications ofvarious degrees of “embodiment” of a car by our brain.
In other words, our brain quickly builds a sort of internal
model of certain relevant aspects of a car, so that, in somesense, one can say that “it (the car) becomes part of ourbody”. For these mechanisms to be efficient, for safe,
comfortable and pleasant driving, the haptic “feel” of thecommands of a car are thus essential. The progressive
stiffness, the appropriate amount of viscous damping, theend-stop characteristics of steering wheel, accelerator andbrake pedal are essential elements in this context, theyare (or should) accordingly be fine tuned very carefully.
Although to a lesser degree, such reflections apply also
for secondary input devices such as knobs, buttons,
handles, seats, the “feel” of closing the door etc. Major
car manufacturers devote considerable attention to suchquestions.
Other fields of application where haptics is essential
are e.g. the design of input elements such as keyboards,
touch screens, musical instruments, household appliances,
sports equipment. In all these domains, a thoroughanalysis of the haptic properties of the interface can leadto decisive improvements and can sometimes translate
into spectacular marketing successes or failures.

6

Our mind, our image of ourselves as an individual iscompletely determined by the sum of our sensory inputs.
Of these, the various modalities of the haptic senses (in alarge sense) are, in certain ways, more basic than opticaland acoustic data inflow. This is now well acceptedby roboticists as well as by researchers in cognitive
neuroscience.
Research in haptic man-machine interface has
therefore become an active branch of robotics, biomedical
and mechatronics engineering. Besides pure researchinterest, such as evidenced e.g. through the link tocognitive neurosciences, many practical fields stand tobenefit from a better understanding of the interplay ofhuman physiology and mechatronics. As examples, theimportance of haptic man-machine interfaces in robotic
surgery, rehabilitation and assistive devices have beenpresented. The focus on haptics is also crucial in manyother domains, specifically in the automotive industry.
*1 DA VINCI is a registered trademark of Intuitive
Surgical, Inc.
*2 ROBOT SUIT HAL is a registered trademark of
CYBERDYNE Inc.

Acknowledgements
Swiss National Science Foundation SNF, NCCR
“Robotics” program
Dr. Mohamed Bouri, Jeremy Olivier, Simone Gallo, Ali
Sengül, Giulio Rognini EPFL LSRO

References
1)
2)
3)
4)

5)

6)
7)

Vincent Hayward et al. Haptic interfaces & devices,
Sensor Review, 24, 1, pp16-29
Sawako Ariyoshi: The Doctor’s wife, Kodansha America,
Inc (1979)
Haptics for teleoperated surgical systems, Tavakoli et al.,World Scientific ed., 2008 (158 p)
Ali Sengül, PhD thesis EPFL No 5669, 2013: “Cognitive
Neuroscience Based Design Guidelines for Surgical
Robotics”
Giulio Rognini, PhD thesis EPFL, 2014: “Cognitivehaptic interfaces: robotics, sensorimotor processing andbodily self-conciousness”
Bleuler, Bouri: “Haptics in Robotics, Man-MachineInterface and Neuroscience”, in 7)
Pisla, Bleuler, Rodic, Vaida, Pisla (editors): “New Trends
in Medical and Service Robots”, theory and integrated
applicatrions, Springer (ed.), 2014

JTEKT ENGINEERING JOURNAL English Edition No. 1012E (2015)