High-Resolution Tactile Sensor Using the Movement of a Reflected Image
Satoshi Saga∗

Takeshi Morooka†

Hiroyuki Kajimoto‡Susumu Tachi§

Univ. of Tokyo

Univ. of Tokyo

Univ. of Tokyo

Univ. of Tokyo

A BSTRACTIn recent years, many tactile sensors have been developed with theadvancement in robotics. For example, there are sensors that measure the contact state or force distribution. They are very useful,but the resolution of the measurement is still inferior as comparedto that of a human. Thus, we propose a new type of optical tactilesensor that can detect surface deformation with high precision byusing the principle of optical lever. We construct a tactile sensor
that utilizes the resolution of a camera to the maximum by usingtransparent silicone rubber as a deformable mirror surface and taking advantage of the reflection image.
Keywords: tactile sensor, optical lever, reflected image, transparent silicone
1

I NTRODUCTION

In recent years, with the advancement in robotics, many tactile sensors have been developed. Several tactile sensors are available inthe market: some of them include a force sensor with 6 degreesof freedom that can measure the force at one point. Another sensor can measure the distribution of the contact state or one that canmeasure the force distribution [1].
The disadvantage of these distribution–type force sensors is thatthe number of sensor units with dense of wiring is excessive. Eachsensor unit is arranged in close proximity to the measurement surface in order to allow a small sensor to be individually distributed,
and the wiring that gathers information from a unit is also individually wired. Therefore, a sensor itself cannot avoid deterioration due
to the stress of repeated measurement; further, the assembly of thewiring is complicated.
Some optical sensors [9, 4] such as the distribution–type opticaltactile sensor exist at the study level. They provide fine examples by
the use of which a unit and the corresponding wiring can be eliminated from the measurement surface by using a camera as an opticalsensor. However, as it measures the motion of markers that are embedded in the elastic body, the sensor resolution is determined by
the markers, and not by the resolution of the camera. Therefore, itdoes not fully utilize the resolution of the camera.
On the other hand, several techniques, which have not been so
far used for tactile sensors, to measure the shape of an object havebeen studied. For example, there is a method that uses a moire interference fringe pattern or slit light when the object is a dispersion
surface [2, 6]. In addition, there is an exact measurement methodthat uses a laser and a collimator [3]. One method uses the specularsurface of water [7]. However, in these methods, it is difficult toimprove the resolution of time, hence; these techniques are limited
to static objects.
∗ e-mail: saga@star.t.u-tokyo.ac.jp† e-mail: t40569@mail.ecc.u-tokyo.ac.jp‡ e-mail: kaji@star.t.u-tokyo.ac.jp§ e-mail: tachi@star.t.u-tokyo.ac.jp

Thus, we examine a sensor that uses an optical lever. An opticallever is a technique that magnifies the displacement by using thecharacteristic of reflection. The optical lever technique is mostly
used in minute domains, like sample measurements under a microscope. In addition, the deformation of a glass surface was measured
in the study [5]. The object measured in this technique is static glasssurface and it premises a recursive calculation. Moreover, this approach measures the deformation of the glass surface itself and does
not measure the contact object. We proposed a sensor with a newsystem that takes advantage of an optical lever, and measures a reflection image in order to use the resolution of a camera; further, theviewpoint of the tactile sensor measured the contact object. In otherwords, we construct a tactile sensor by placing an optical lever anda flexible mirror surface together, and use this combination.
Transparent silicone rubber is used as the flexible mirror surface.
Because of the use of silicone rubber and the distribution of the refractive index at the air interface, the boundary surface has a reflection characteristic similar to that of a mirror surface. We constructa sensor with a flexible mirror surface by using this mirror surface
reflection characteristic (figure 1).

Figure 1: Section of a sensor

2

P RINCIPLES

In our sensor, the optical lever plays an important role. The aim isto develop a flexible mirror surface in order to realize this opticallever. We can acquire a reflection image in a mirror surface by using
the above mentioned properties and obtain the deformation of themirror surface in itself using geometrical optics by measuring thedisplacement. Further, we explain each of these elements in detail.
2.1 A Reflection Condition
The optical lever technique is mostly used in minute domains, likesample measurements under a microscope. By irradiating the tipof cantilevers with lasers and using the characteristic of reflection,
an optical lever is used to magnify the displacement of a reflection
surface. In addition, the deformation of glass surface was measured
in the study [5].
Transparent silicone rubber is used as the flexible mirror surface.
With regard to the boundary between silicone rubber and air (figure2), the refractive indexes are denoted by ns and na , respectively,and the refraction angles by φs and φa , respectively. Further, theincidence angle is indicated. When the distribution of the refractive
index at the boundary of silicone rubber and air satisfies equation
3, total reflection occurs; further, this boundary surface assumes thereflection characteristic of a mirror surface. We construct a sensor
with a flexible mirror surface by using the above mentioned mirror
surface reflection characteristic.

Figure 3: Geometrical optics
Figure 2: Snell’s law

ns sin φs

=

φs

=>

na sin φanaarcsin( sin φa )nsna
arcsin( )
ns

(1)
(2)
Figure 4: Geometrical Optics (Part)(3)

2.2 Deformation of the Reflection Image
In some optical tactile sensors, a camera is used as an optical sensor[4]; the sensor obtains the deformation by tracing a marker. However, the resolution of this sensor is limited because of the resolution of the marker. We propose a tactile sensor for the new systemthat uses a reflection image in order to use the resolution of thecamera to the maximum possible extent. We employ it as a tactile
sensor by combining an optical lever and a flexible mirror surface.
By arranging the image pattern, camera and the transparent silicone
rubber similar to the arrangement in figure 1, light dispersed fromthe image pattern is reflected from the silicone rubber boundary tothe camera. If the contact object touches the silicone rubber, thesilicone rubber boundary will be deformed. The reflection surface
results in deformation, and the reflection image of the image patternis deformed. We measure the deformation of the reflection surface
itself by solving an inverse problem using this deformation. Therefore, the entire reflection image contains information; as a result,
the resolution of the camera can be utilized to the maximum extent.
However, we only use the characteristic point of the captured image
in this study, we can use all the points in the captured image in theform of information for reconstruction.
2.3 Geometrical Optics
When the captured pattern is arranged as a planar as shown in figure
3, the surface with this pattern is referred to as a pattern surface p.
The angle between the pattern surface p and the reflection surface
q is represented by α0 , and the angle between the captured surface
r and reflection surface q by β0 .
We define the angle at each point on the reflection surface q as
θ . We assume that after some deformation Q1 , which is at a distance l from P0 , is projected at a distance d and inclined to theangle θ1 with respect to the surface q0 before deformation. Similarly, after the deformation Q2 , which is at a distance of l + ∆l fromP0 , is projected at a distance d + ∆d and inclined to an angle θ2 .

Further, we assume that an image is formed when P1 → Q1 → R1
and P2 → Q2 → R2 ; the ray group captured by the camera is assumed to incline to the pattern surface at angles β1 and β2 and atangles α1 and α2 to the captured surface. If the degree of leaningof the reflection surface in Q1 and Q2 changes continually similarto θ1 → θ2 , ∆θ , height displacement ∆d and the length of Q2 QH
can be expressed as follows:∆dZ ∆l∆θl)dl∆l0∆lcos θlog(||)∆θcos(θ + ∆θ )==tan(θ +(4)(5)

In addition, based on the symmetry of an incidence angle and thereflection angle on the reflection surface, the following eqations areobtained:
6P2 Q2 QH

α2===

6 Q Q Q − 26 Q Q Qn
H 2 R
H 2
β2 − 2θ2β2 − 2θ2(6)(7)(8)

Similarly,

α1 = β1 − 2θ1(9)Next, we express m = P0 P1 , w = R0 R1 with l. Based on the sinetheorem at 4P0 P1 Q1 in figure 4, we get

γ = arctandl√
l2 + d2m=πsin( 2 − (α0 − α1 )) sin( π2 − (α1 − γ ))

(10)

(11)

m=

pl2 + d2cos(α1 − γ )cos(α1 − α0 )(12)Similarly, we get

w=(13)d [Pixel]dγ 0 = arctan 0lpcos(β1 − γ 0 )l 02 + d 2cos(β1 − β0 )(14)




2.4 Reconstruction of the reflection surface
We can measure m, w, β1 and β2 using our sensor; L0 , α0 and β0 areknown. The quantity required is the distribution of θ and d. We
can construct ∆d if ∆θ and ∆l are known from equation 5; further,
we can reconstruct θ and d if ∆θ , ∆d are known. In this case, theexpressions 12 and 14 cannot be solved for unknown values of θ →θ + ∆θ , l → l + ∆l since there are only a few equations, however,in this situation it is not solved for a number to be unknown asθ → θ + ∆θ , l → l + ∆l in expression 12,14 because there are a fewequations.
Here, we define∆α ≡ α1 − α0(15)From the equation 12, we get
m2 cos2 ∆α − (l 2 + d 2 )(cos2 (α0 + ∆α )) = 0(16)When ∆α is expressed as γ = 0, ∆α ' 0 from the second Taylor
series expansion, equation 14 can be expressed as follows:∆α =22−(dv + l ) cos α0 sin α0 u 2 2 2 2
 u (d + l ) cos α0 sin2 α0 u ±t −(m2 − (d 2 + l 2 ) cos2 α0 )×((−m2 ) + (d 2 + l 2 )(cos2 α0 − sin2 α0 ))(−m2 ) + (d 2 + l 2 )(cos2 α0 − sin2 α0 )∆lncos θnlog(||)∆θncos(θn + ∆θn )Figure 5: Smooth deformation

3.1 Smooth Surface
We assume a curved surface (upside down), as shown in figure 5, as
the reflecting surface and assume that β1 = β2 = β0 , L0 = 1637andπ
α0 = β0 = ; the camera center is placed at infinity.
4We calculate the point Rn (n = 0...N) where the disintegration
point on the pattern surface Pn (n = 0...N) is reflected onto the captured surface by the reflection surface. In this manner, by calculating Pn and Rn and the equation 19, we solve an inverse problem
and obtain the angle distribution θn and depth distribution dn as anapproximate distribution of d can be traced almost precisely.

(17)

Figure 6: Simulation result of θ(18)(19)

Since we assume d = 0 at l = 0 and l = L0 , we get ln and θn fromequation 12. We obtain ∆θn from the given value of θn and calculate∆dn+1 equation 19. We get αn , dn , and the distribution of α and dby iterating this calculation step-by-step.
3l [Pixel]Based on this, we can express equation 5 as follows:∆dn+1 =



From the equations 9,15 and 17 we can express θ1 with the known
parameters and measured values.
By defining Q1 in equation 5 as the nth characteristic point, anddefining ∆d as ∆dn , from the continuation property of a boundary
surface, we assume∆dn ' ∆dn+1

k [Pixel]

S IMULATION

Based on the geometrical optics elaborated in the above mentioned
paragraph, we confirm the precision by using a simulation. Thissensor has 1637 [pixel] square size and deformed size in verticaldirection is 13 [pixel] (figures 5 and 10).

We examine an error in the true value in detail. Figure 8 showsthe error distribution of θ , and figure 9 shows the error distributionof d. An error rate (the error/truth value) is suppressed to approximately 1% in the entire area.
The value at l = 0 is near the true value in the series of rises, andl = L0 is near the true value in the series of drops.
3.2 Needle-Shaped Surface
We assume a needle-shaped surface with an upside-down shape, asshown in figure 10, as the reflecting surface; further, β1 = β2 = β0 ,πL0 = 1637 and α0 = β0 = ; the camera center is assumed to be at
4infinity. The calculation results are shown in figures 11 and 12.
A solid line represents the shape of the original reflection surface; − represents the result of the summation of ∆l and ∆d froml = 0 in a series of rises. 4 is the result of the summation fromd [Pixel]k [Pixel]l [Pixel]

Figure 10: Needle-shaped deformation

Figure 7: Simulation result of depthl = L0 in a series of drops, and ° is average of − and 4. It can beobserved that the distribution of d can be traced almost precicely.

Figure 11: Simulation result of θ
Figure 8: Simulation error result of θ

We examine the error in the true value in detail. Figure 13 showsthe error distribution of θ , and figure 14 shows the error distributionof d. An error rate (the error/truth value) is suppressed to approximately 1% in the entire area. The value of l = 0 is near the true
value in the series of rises, and l = L0 is near to the true value in theseries of drops.
In addition, in comparison with the result of the smoothly curved
surface, significant difference is not observed. In other words, thisshows that our sensor can measure the shape without any problem
even if differentiation is impossible in the plane of reflection.
4M OUNTING

Based on the simulation in the above mentioned paragraph, webuild a real system. By using an implemented sensor, we measure
the real tranformation in a real environment.
4.1 Silicone Rubber with Image PatternFigure 9: Simulation error result of depth

We use the addition-polymerization-type silicone rubber(KE109A,B[8]).
We prepare a triangular pole-shaped flaskwith transparent acryl and pour silicone rubber into it and perform
a lap reaction. After lap reaction, we exfoliate with one acryl board
and make a sensor surface. One of the left acryl surfaces becomea pattern surface and another one become a captured surface. In
this case we use a lattice-patterned paper as a pattern surface. Inthis manner, we prepare a sensor having a flexible mirror surface(figure 15). We install a camera to capture the flexible mirror

Figure 12: Simulation result of depth

Figure 15: Silicone rubber with a lattice patternsurface adequately and fix it (figure 16).

Figure 16: Overview of the sensor

Figure 13: Simulation error result of θ5

Figure 14: Simulation error result of depth

E XPERIMENTWith the implemented sensor, we measure the real tranformation
in a real environment. In this case, we place the tip of a pencil onthe sensor face (figure 17) and measure the status when we push it
down by about 0.2 [mm].
We show the acquired image before and after the deformationin figures 18 and 19. Here we use a lattice pattern as the image
pattern. We took association between one line of central verticallattice points of two pieces of images. By using the equations 9,15
and 17 a shape is reconstructed.
Figure 20 shows the series of rises and the series of drops. The
scale of the image is 1 [pixel] ' 0.1 [mm] because the acquired
image size is 625 × 392 [pixel], the lattice pattern has a width of 2.5[mm] width, and there is 16 pieces of lattice in vertical orientationin the image. Then, the peak value in figure 20 is almost near thetrue value. An error rate (the error/truth value) is suppressed toapproximately 10% at the peak area. The value of l = 0 is near thetrue value in the series of rises, and l = L0 is near the true value inthe series of drops. The solution of increasing accuracy is considerd
as future work.

6Figure 17: Deformation overview

C ONCLUSION

In this paper, we proposed a tactile sensor that employs a new
method that utilizes the principle of optical lever and a flexible reflection surface. We constructed a tactile sensor that took advantage of the reflection image whose deformation was detected withhigh precision using an optical lever; the sensor also took sufficientadvantage the resolution of a camera by using transparent silicone
rubber as a flexible mirror surface.
Through a simulation, we showed that the reconstruction of aplane of reflection from an image was possible. The reconstrucion
is carried out by using geometrical optics and by an appropriate
approximation in the constituted tactile sensor. In addition, we actually produced a prototype made of silicone rubber and showedthat shape restoration of a plane of reflection was possible.
Our proposed technique combines the optical lever and a flexible surface of reflection. Therefore, the modification of a patternsurface and a captured surface according to the measuring object
is possible. In other words, we can arrange a laser, light emmision
diode (LED), or liquid crystal display (LCD) on the pattern surfaceand a camera or photo detector (PD) on the captured surface. Inthe future, we wish to examine these combinations, in particular,
dynamic pattern generation using a liquid crystal display.
R EFERENCES

Sampling point
for reconstruction

Figure 18: Before deformation

Sampling point
for reconstruction

Figure 19: After deformation

Figure 20: Result of depth[1] Nitta Corporation. Flexi force.
http://www.nitta.co.jp/product/mechasen/sensor/
tactile_top.html.
[2] Miike Hidetoshi, Koga Kazutoshi, Yamada Takehito, Kawamura Tatsuji, Masaaki Kitou, and Takikawa Naoshisa. Measuring surface shapefrom specular reflection image sequence - quantitative evaluation ofsurface defects of plastic moldings. Japanese Journal of AppliedPhysics, Part 2, 34(12A):L1625–L1628, Dec 1995.
[3] Yamaguchi Ichirou, Yamamoto Akihiro, and Yano Masaru. Surface tobography by wavelength scanning interferometry. Optical Engineering,39(1):40–46, Aug 1999.
[4] Kazuto Kamiyama, Kevin Vlack, Hiroyuki Kajimoto, Naoki
Kawakami, and Susumu Tachi. Vision-Based Sensor for Real-Time
Measuring of Surface Traction Fields. IEEE Computer Graphics &Applications Magazine, pages 68–75, 2005.
[5] Jürgen H. Massig. Deformation measurement on specular surfaces bysimple means. Optical Engineering, 40(10):2315–2318, October 2001.
[6] Baba Mitsuru, Konishi Tadataka, and Handa Hisashi. Shape measurement of columnar objects with specular surfaces by slit ray projection
method. Systems and Computers in Japan, 33(4):50–60, April 2002.
[7] Howard J. Schultz. Specular surface stereo: a new method for retrieving
the shape of a water surface. Proceedings of SPIE - The International
Society for Optical Engineering, 1749:283–294, 1992.
[8] Shin-Etsu Chemical Co., Ltd. 2 liquid type RTV rubber KE109.
[9] Kouichi Yamada, Kenji Goto, Yoshiki Nakajima, Nobuyoshi Koshida,and Hiroyuki Shinoda. A sensor skin using wire-free tactile sensing
elements based on optical connection. In SICE 2002. Proceedings of the41st SICE Annual Conference. Soc. Instrument & Control Eng. (SICE),Tokyo, Japan.