Control Method for a 3D Form Display
with Coil-type Shape Memory Alloy
Masashi Nakatani1, Hiroyuki Kajimoto1, Kevin Vlack2, Dairoku Sekiguchi1, Naoki Kawakami1 and Susumu Tachi1
1Graduate School of Information Science and Technology

2

Graduate School of Interdisciplinary Information Studies
The University of Tokyo
Hongo 7-3-1, Bunkyo-ku, Tokyo, Japan{nakatani, kaji, vlack, dairoku, kawakami,tachi} @star.t.u-tokyo.ac.jp
Abstract - We previously proposed a new 3D form display
actuated by shape memory alloy (SMA), which is capable ofdisplaying large scale objects sequentially. Based on ourdevised method, our display uses a 16×16 array of pin-rods
with 30mm stroke at 5mm intervals. In this paper, we discuss
the control method of the shape memory alloy with a simple
matrix drive circuit. This type of drive circuit can dramaticallyreduce the number of switch. In addition, we propose a novel
approach using linear programming, where the simulated time
to display a whole shape is reduced by about 50% or morewith a large number of actuators.
Index Terms – 3D Form Display, Shape Memory Alloy,
Simple Matrix Drive Circuit, linear programming, Visual Tactile
Display.
Fig. 1 The example of presenting shape (Heart shape)

I. INTRODUCTION
We previously proposed a novel 3D form display that iscomposed of an array of pin-rods actuated by coil-type
shape memory alloy (SMA)[1]. The goal of this display is topresent a large scale object with subtle expression based on
densely arranged pin-rods. The stroke length for each pinrod should be long enough to display an arbitrary shape likea geometrical feature or a human face. The movement of
this display should be smooth enough to seamlessly change
between displayed shapes. In addition, the number of pinrods composing the whole display should be large enoughfor presenting very large objects like geographical features.
We devised a method that satisfies these desired
specifications, and developed an actual display composed of
256 pin-rods. Thus we can display arbitrary shapes asshown in Fig. 1. We can change the presented shape easily
and touch the object’s surface.
Perhaps the most complicated aspect of an SMA
actuated display is the control method. Because of its
dynamics, shape memory alloy cannot be driven instantly
without a high applied voltage, and then it is difficult tocontrol each pin-rod height with precise accuracy.
Therefore, we need to develop a novel driving algorithmwith a simple matrix drive circuit that is suitable for theeccentricities of SMA. In this paper, we overview thedeveloped prototypes of the 3D form display, then describe
our devised method, and finally evaluate its efficacy
through simulation.

II. 3D FORM DISPLAY OVERVIEW
The proposed system to realize a 3D shape display iscomposed of four key elements: the actuator, pin-rod,
positional sensor, and control circuit. We discuss eachelement in detail in the following sections.
A. ActuatorTo achieve a dense display, the actuator for each pixel
must be compact, so we use a kind of Shape Memory Alloy(SMA), material that can be stretched or deformed from its
original shape, but would spontaneously return to its
original shape when heated. However, conventional SMA
can only extend its length by about 2-4%, which is notenough to realize our goal of a long stroke for each pin-rod.
Therefore, in our study, we use Coil-type ShapeMemory Alloy (C-SMA) called Bio Metal Helix 200(manufactured by Toki Corporation Co-Ltd.). This C-SMA
is composed of SMA formed into a coil, and can therefore
be extended up to twice its original length, and contracted toits original length by heating. A single strand of C-SMA isvery thin (0.85mm in diameter), so it is possible to arrange
densely in a pin matrix, while it has enough power toproduce a 40-60gf force. We contract the C-SMA by simply
applying an electric current to produce Joule heat. C-SMA
is easy to extend when it is at room temperature, so we
attached a spring to one end of the C-SMA, whichconstantly pulls the C-SMA, as shown in Fig. 2.
The range of motion depends on the C-SMA length. In
the preliminary experiment for a 1m rod (42.1gf) with C-SMA (750mm), the pin-rod is actuated at 500mm, and alonger length is possible for larger device units[1].
B. Pin-rod Configuration
The design of the pin-rod is just as crucial as theselection of the actuator for assembling a display with highdensity.
Fig. 2 illustrates our proposal for a pin-rod design. The
pin-rod is composed of a cylindrical plastic pipe, whichcontains the C-SMA and circuit components as depicted in
the figure. The C-SMA is fan-cooled, and it is important foreach pin-rod to have sufficient aeration, so it has two pairsof slits facing each other, placed at right angles. This slit
arrangement serves as a guide rail for motion, an entry point
for current, and a window to control airflow.
Each pin-rod contains a diode for eliminating
unintended loop current caused by simple matrix drive
circuits[1][2][3].
C. Positional Sensing Method
Because the C-SMA itself only produces slightly
detectable information related to its contraction, it isnecessary to obtain the current position of the pin-rods withan external sensing method. Although it has been reported
that the resistance change of simple SMA can be used for
detecting its length[4], the actuator we use is coil formed,
thus the model is changed after it has been actuated a few
times. Therefore, we opted to pursue other methods to sense
the actual position directly.
There are many ways to detect position, such as the useof a linear encoder[1], the inductance change of the coil[5],and the use of capacitance change[6]. However, the timeand cost to construct the display increase dramatically byequipping a sensor for each pin-rod, and even very compact
sensors occupy finite space, which greatly complicates thedevelopment of dense pin-rod arrays.
To solve this problem, we devised a method for
detecting the pin-rod height through vision-based
techniques. Each pin-rod has a colored marker on thebottom. The marker color is selected such that no adjacentpin-rods have the same color, in order to clearly distinguisha pin-rod from its neighbors in a captured image. We place a
CCD camera on the inaccessible lower side of the display
beside the pin-rods to capture their movement, as shown in
Fig. 3.
In order for all markers to be visible to the camera, thelength of the pin-rod increases according to its row number,and the length increments are determined depending on the
maximum stroke of each pin-rod, which is given by the slit
length mentioned in previous section. In this fashion,
camera images forwarded to a PC can be used to computeall marker positions with a fast tracking algorithm, and thereceived state is processed as a height map of each pin-rod.
Based on this height map, an appropriate control signal isissued according to the algorithm described in next section.

Fig. 2 Actuator & pin-rod configuration

D. Driving Circuit
We adapted a well-known simple matrix type circuit todrive electrical current through each pin-rod. Fig. 4 shows
our adopted method, with linear switch arrays positioned
perpendicularly at opposing ends of the pin-rod matrix.
With this method, the amount of information required todrive the system is proportional to 2N, where N is thenumber of pin-rods on each side of the array. This contrasts
with the N2 complexity for circuitry that drives each pin
individually, which will be increasingly important as thesize of the array increases[1][2][3].

Fig. 3 Position acquiring method
Upper: Principle of measurement system
Below: Actual implemented system

Fig. 4 Matrix drive circuit

E. Developed Prototypes
Adapting these devised methods, we developed several
prototypes, which could successfully display arbitrary
shapes. Fig. 5 shows the system block diagram wedeveloped in this project. A PC outputs the PWM signal for
driving a FET as a switch, and adjusts the time-averaged
voltage applied to the C-SMA. We use a VGA 640×480
color camera connected to the PC with an IEEE1394 cable,
and the PC computes the pin-rod positions and feeds back acontrol signal to the C-SMA in the pin-rods. Each pin-rod is
operated by this driving-measurement system, so it ispossible to control an arbitrary number of pin-rods, given
sufficient processing power. Pipes made of styrol resin(diameter: 4mm) were used as pin-rods, which were placed
at 5mm intervals. The height of each pin can be controlled
to 0.4mm intervals, granting the capability to display finetextures. We use a cooling fan to easily extend the C-SMA
for each pin-rod.
This prototype was also designed to be touchable. As
shown in Fig. 6, the pin-rod positions are detected in realtime, so the corresponding CG model changed according tothe operator’s applied force. Unfortunately, C-SMA cannot
produce enough power to present a sense of force, and the
pin-rods were easily pressed down by hand. However, thismalleability can be used as a form of 3D CAD, in which CG
modelers may sculpt the displayed shapes directly by theirown hand, without needing a conventional computerinterface like a pointer device.
A second prototype was developed with the pin-rod
array submerged in a container of oil to improve theresponse speed (Fig. 7). The prototype had no positional
feedback, so pin-rods could only adapt an “on-off” state.
Each pin-rod was made of bakelite, with the C-SMA
sandwiched between thin bakelite plates. The pin-rod
interval was also 5mm, and the pin-rod diameter was 4mm.
The oil is cooled by a Peltier effect element placed at thebottom of the container, and is admixed to keep the oiltemperature uniform. In this condition, the pin-rod can be
actuated up to about 1 Hz with over 30 volts. Using a linescanning method, the power consumption with oil cooling
was about 1000 watts.

Fig. 5 Sytem overview

Fig. 6 Interaction with 3D form display
The model displayed on the screen is changed according to theoperator’s applied force.

states “open” and “closed”, respectively. According to thecircuit in Fig. 8, the product:⎡ u1v1 L u1vn ⎤rT r ⎢
A=u v =⎢ M O
M ⎥⎥⎢⎣unv1 L unvn ⎥⎦(1)

depicts which pin-rods are actuated instantaneously by agiven control signal pair. A has many variations, forexample, when the dimension of u and v is 4, the numberrT rof unique combinations u v can take is 225 + 1 (notrT rFig. 7 Oil injected prototype for improving response time
Character “U” is displayed rapidly.

III. CONTROL METHOD
A. Problem and motivation for a new driving method forsimple matrix drive circuit
In the previous section, we described our developed
system, which can present arbitrary shapes. However, there
is one crucial problem in using a simple matrix drive circuit
namely dynamics of SMA. It takes a long time to actuate
every pin-rod to present one shape (about 5 – 10 seconds),
which is undesirable for a form display that changes fromone shape to another.
We have driven the 3D form display with aconventional line-scanning method commonly used forsimple matrix devices such as a liquid crystal display. The
line-scanning method iterates over every row in the array,and the signal lines are switched to contract the C-SMA of
the pin-rods in the corresponding columns. This procedure
is repeated until the pin-rod arrives at its desired position.
Although this scheme appears reasonable, it does not work
well in practice, because of the dynamics behavior of CSMA, which changes its length about 30mm. If we adopted
this scheme, it would take a long time to display only one
shape because C-SMA cannot contract instantaneously.
The purpose of the scheme we propose here is tominimize the necessary time to actuate all pin-rods byaccounting for the complex dynamics of each C-SMA coil.
We seek a strategy to minimize the total time bymaximizing the number of C-SMAs actuated with one set ofcontrol signal.
To state the problem mathematically, we first define
several variables. We consider the problem of determining
the optimal approach to drive an n × n matrix to match ashape S, represented as an n × n matrix of integer heightvalues. According to the matrix drive circuit, control
signals are realized as the dynamic transmission of two n ×
1 bit-vector control signals u and v.
The scanning line switch is denoted as:ru = (u1 , u 2 , L , u n )The signal line is denoted as:rv = (v1 , v 2 , L , v n )The components of each vector ui and vi areconstrained to be within the set {0,1}, referring to switch

44=256 since u v will be the zero matrix for 30 signal
pairs), as shown in Fig. 9. When the dimension is 6, thetotal number of possible combinations is 3970, and when
the dimension is 8, the total number is 65026.
We assume that the pin-rod does not descend after
moving upwards until the displayed shape is completed.
We define a new n × n matrix B that represents theremaining change in height to display a shape. Here werr
reframe equation (1) by defining a new vector a and bcomposed of the components of respective matrices A andB, as follows:ra = [u1v1, u2v2 ,L, u1v1, u2v1,L, unvn ]rb = [ B11 , B12 ,L, B1n , B21 ,L, Bnn ]

(2)
(3)

r
Here we define the time t applied to a switch state a ,rwhere a is one of the possible switch state combinations.
rThen, the vector b is represented as:
r
r
r
r
r
b = t1a (1) + t2 a (2) + L + tm a (m) = Aˆ t

(4)

r r
r
where Â is defined as Aˆ = [a (1), a (2), L a (m)] . The
maximum value m is the same as the total number ofpossible combinations. For example, when n=4, the value ofm is up to 226 (Fig. 9).
Again, the desired goal is to minimize the total time todisplay a shape. In other words, we must minimize
t1 + t2 + Ltm . Consequently, the goal is represented as a linear
programming problem as follows:

subject tor r
1 ⋅ t → min

(5)

r
r
br = Aˆ t
t ≥0

(6)
(7)

After this formulation, we can solve this problem byusing standard linear programming library. We show asimple example for an 8×8 matrix. The height map can berealized with the switch combination shown in Fig. 10. In
this example, linear programming method can realize thedesired height map in half time as many as line-scanning
method.

Fig. 8 Conventional line-scan method overview
Here we difine horizontal line is Scanning line, and vertical line issingnal line.

Fig. 9 Possible combination of pin-rod state (matrix size is 4 x 4). Bluesquare means “pin-rod is OFF,” and red square means “pin-rod is ON.”

B. Evaluation
In this section, we evaluated the proposed algorithm bycomparing with conventional line-scanning method
We simulated the time that is needed for arbitraryheight map, which is given a pin-rod matrix. We consider
square matrices with sizes ranging from 1×1 to 8×8. The
goal height map was given as a uniform random
distribution. Each height value was an integer from 0 to100.
We executed the simulation 1000 times for each matrix
size. The result is shown in Fig. 11. With line-scanning
method, the average required time increases proportionallyto matrix size, which is naturally expected result of the linescanning method. On the contrary, with the linear
programming method, the average of required timeincreases according to the square root of matrix size. The
standard deviation of required time for the linear
programming method does not dramatically change. This
result indicates that as the size of matrix increases the linear
programming method can display arbitrary shapes in ashorter time than with the line-scanning method. When thematrix size is only 8, the required time is reduced by anaverage 56%, and at a maximum of 65.5% in our simulationdata.
IV. DISCUSSION AND CONCLUSION
In this paper, we presented a 3D form display using a
dense array of pin-rods actuated by SMA, and discussed its
control method for a matrix drive circuit. With our proposed
method, we can display arbitrary shapes with significantly
less time than with a conventional line-scanning method.
We showed that our method is effective, and in 8 by 8
matrix, it can reduce the required time by more than 50 %.

Fig. 10 One example of representing a circle with 10 switch states foran 8 x 8 pin-rod matrix by applying linear programming, contrary to 8
switch states for line-scnanning method. The number below eachswitch state indicates how long each switch state is applied. The total
time for arriving desired height is shorter with linear programming
method.

Fig. 11 Simulation result of matrix size versus required time
The matrix is assumed to be square.

Fig. 12 One example of future applications
Pin-rods matrix presents a rough shape, and a precise texture is
projected on to it. With this device, it is possible to see and touch thepresented object with sense of physical presence. This application can beused as a kind of video conference system for displaying the presenter’s(face) movement or dynamic demonstrations.

In a future paper we shall discuss the implementation of thiscontrol method for an actual device. Although we have
simplified the situation by neglecting the C-SMA’sdynamics, we must consider the dynamic behavior of CSMA, including its hysteresis and cooling condition. In our
next work, we will take the behavior model into account.
The fundamental problem that already arose iscalculation time. While this approach always yields the
optimal result, a brute force approach to the combinatorial
optimization problem is infeasible for real-time display orfor matrix sizes much larger than 8. To realize this type of
3D display with real-time motion or larger matrix sizes,
there are two alternatives. The first is to decompose thelarger matrix into blocks, and apply the linear programming
method for each block. However, the benefit of linear
programming over the line-scanning method would not bevery significant for large displays. The second alternative isto obtain an approximate solution to the combinatorial
problem based on heuristics. For instance, a method basedon image compressing can be applicable, in which theheight map corresponds to the intensity of a grayscale
image. One possible candidate is the so-called SemiDiscrete Decomposition (SDD), an iterative greedy
algorithm that represents an arbitrary matrix as the weighted
sum of outer products of vectors whose elements areconstrained to the set {–1, 0, 1}[7]. This algorithm wasshown to compress grayscale images by an order ofmagnitude and is quite fast. Further, it can be adapted to our
problem rather easily, as we shall describe in a future paper.
Therefore, we can choose from the two ways, a strict linear
programming method to obtain an optimal solution offline,
or a fast but sub-optimal method for real time control.
The scalability of this display is perhaps the mostdisputable issue. In our prototype, we realized a 16 by 16
pin-rods matrix, and we believe it is possible to construct alarger system up to 32 by 32 pin-rods matrix with our
proposed method. When the number of pin-rods isincreased, one problem will arise. The problem is the power
consumption of the system. When the C-SMA is actuatedusing PWM and relaxed by air-cooling, the power
consumption is only about 200 Watts. However, with thecooled oil-injected condition for fast response, it has
significantly higher power consumption, roughly five times
as much. Therefore, we must consider the maximum power
supply as a constraint in the adopted driving method. With
the line-scanning method, it is difficult to take thislimitation into account for driving the system. On the other
hand, with the linear programming method, the maximum
performance of the power supply can be taken into accountas an inequality condition of linear programming.
As a near-future application, we will improve the visual
aspect of the display. If we use optical fiber as a pin-rod,
and project visual image or real-time movie from the bottomof the pin-rods matrix, it would become actual 3D television(Fig. 12). The display can also be used as a street
advertisements or showcases. Since this display istouchable, and image is rear projected through optical fiber
so the image is not disturbed by the operator’s hands.
Finally we will obtain an ultimate 3D shape display, bothvisually and haptically. This advantage can be used where
natural representation is needed, especially in the field ofVirtual Reality.

REFERENCES[1] M. Nakatani, H. Kajimoto, D. Sekiguchi, N. Kawakami, S. Tachi, “3DForm Display with Shape Memory Alloy,” in Proc. of 13th
International Conference on Artificial reality and Telexistence (ICAT),Tokyo, pp. 179-184, December 2003.
[2] S. Mascaro, H. H. Asada "Design and Control of Vast DOF Wet SMA
Array Actuators," in Proc. of IEEE International Conference onIntelligent Robots and Systems, Las Vegas, pp.1992-1997, Oct 2003.
[3] K. Cho, B. Roy, S. Mascaro, and H. H. Asada, “A Vast DOF Robotic
Car Seat Using SMA Actuators with a Matrix Drive Systems,” in Proc.
of IEEE International Conference on Robotics & Automation, New
Orleans, pp. 3647-3652, April 2004.
[4] K. Ikuta, M. Tsukamoto, and S. Hirose, "Shape Memory Alloy Servo
Actuator System with Electric Resistance Feedback and Application for
a Endoscope," in Proc. of the IEEE 1998 International Conference on
Robotics and Automation, Philadelphia, pp. 427-430, April 1988.
[5] K. Nagai, R. Fujimoto, S. Miyawaki, N. Oshie, S. Toda and S.
Sugiyama, Consideration of the Properties of a Small Position Sensor
with Low Temperature Drift," In Proc. of IEEJ the 20th Sensor
Symposium, Tokyo, pp. 205-208, July 2003.
[6] J. Rekimoto, "SmartSkin: An Infrastructure for Freehand Manipulation
on Interactive Surfaces," in Proc. of Conference on Human Factors in
Computing Systems, Minneapolis, pp. 113-120, Apr 2002.
[7] T. G. Kolda and D. P. O'Leary, "Computation and Uses of the
Semidiscrete Matrix Decomposition," Computer Science Department
Report CS-TR-4012 Institute for Advanced Computer Studies Report
UMIACS-TR-99-22, University of Maryland, April 1999.
http://www.cs.umd.edu/~oleary/SDDPACK/.