# Follow the Grass: A Smart Material Interactive Pervasive Display

Andrea Minuto, Gijs Huisman, Anton Nijholt

► To cite this version:

Andrea Minuto, Gijs Huisman, Anton Nijholt. Follow the Grass: A Smart Material Interactive Pervasive Display. 11th International Conference on Entertainment Computing (ICEC), Sep 2012, Bremen, Germany. pp.144-157, 10.1007/978-3-642-33542-6_13. http://dx.doi.org/10.1447/icec12_144

HAL Id: hal-01556131

https://inria.hal.science/hal-01556131

Submitted on 4 Jul 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Follow the Grass:
a Smart Material Interactive Pervasive Display

Andrea Minuto, Gijs Huisman, and Anton Nijholt

Human Media Interaction Group, University of Twente
PO Box 217, NL-7500 AE Enschede, The Netherlands
{a.minuto,gijs.huisman,a.nijholt}@utwente.nl

**Abstract.** Smart materials offer new possibilities for creating engaging and interesting forms of interaction and ways of displaying information in a material way. In this paper we describe Follow the Grass, a concept of an interactive pervasive display for public spaces. The display will be built up out of a number of blades of grass that are actuated in eight directions using nitinol muscle wires (i.e. a shape-memory alloy). A Microsoft Kinect-based tracking system is employed to detect users' presence. Follow the Grass can be used for entertainment purposes by displaying animations through movement of the grass, as well as for indoor way-finding and ambient persuasive guidance. We present a number of scenarios with varying scales of interaction, and different applications, followed by a description of the initial hardware design of a single blade of grass and its actuated root. We will give a description of the tracking system, and how it tracks users and is capable of identifying individuals. Finally we will provide suggestions for the further development.

**Keywords:** Smart material interface; quiet computing; ambient; organic; display.

## 1 Introduction

For two decades researchers and designers have actively explored new methods of physical interaction with computer systems [7]. The seminal work of Ishii and Ullmer [11] helped spur the development of tangible interfaces; interfaces that allow users to manipulate physical objects that modify digital information. With technological advances, tangible interfaces have started to incorporate smart materials [10]. These materials are characterized by having physical properties that can be altered in a controlled way. Interfaces that incorporate such materials can be referred to as Smart Material Interfaces (SMIs [18]). In essence, where “traditional” tangible interfaces couple physical objects to digital information often displayed on a screen, smart materials allow the digital information to be represented by the physical state of the material. An advantage of this is that the information can be directly embedded into the materials of the physical environment. Furthermore, the use of smart materials allows for new forms of silent, energy efficient and more natural ways of actuating physical output (e.g. Sprout I/O [4]; Lumen [20]).
In this paper we will present a concept of an interactive pervasive display that uses nitinol shape memory alloy (SMA) wires to actuate an artificial blade of grass in eight directions. These blades of grass can be thought of as physical pixels, and, when multiplied, can form a physical ambient display. Such a display can be employed for entertainment, indoor way-finding, or ambient persuasive guidance.

We will first review related work on ambient displays, ambient persuasive guidance and smart materials. Then we will present a detailed outline of the “Follow the Grass” concept, present a number of scenarios, and offer a description of the hardware prototype of the blade of grass and actuated root, and software design of the tracking system. We will conclude by providing suggestions for the further development of the hardware and software.

# 2 Related work

## 2.1 Ambient technology

The interest in Tangible Interaction [11] is part of a development in HCI research that is moving away from the typical “window, icon, menu pointing device” (WIMP) interaction [13]. Reality-based interaction (RBI) encompasses interfaces and interaction styles that support humans’ existing knowledge about the physical world [13]. In this sense, the vision of *ubiquitous computing* is particularly relevant. Weiser’s view [26] of ubiquitous computing holds that computing can be unobtrusively integrated into everyday objects and environments. This proposes a direct coupling of computation and the “real” physical world, in an integrated natural manner. Furthermore, distributed networks and sensor technology can make such systems intelligent, resulting in objects and environments that can naturally adapt to their users [27]. Examples of such systems being used for presenting information or providing entertainment are plentiful. The ambient ROOM [12] uses lights, sounds and movement to unobtrusively communicate a host of background information, such as the physical presence of others or the activity of an actual hamster. Similarly, Hello.Wall [21] consists of a grid of lights integrated into a wall, information can be presented by changing light patterns, that react to the proximity of the user to the installation. Another example is The Information Percolator [9], which displays information through air bubbles rising up in a number of transparent tubes filled with water. One application of The Information Percolator tracks people’s movement along a corridor and represents this information in the ambient display. A more physical approach to ambient displays is implemented in Super Cilia Skin [22], which is a magnetically actuated display that makes cilia (i.e. individual actuators) move lightly to emulate the physical touch of a finger or a gesture trace. A similar approach has been used in the artistic installation Project Dune (Studio Roosengaard<fnref n="1" />), which is composed of large amounts of fibers that brighten according to the sounds, touch and motion of passing users, accompanying them along the walk.

<footnote n="1">¹ http://www.studioroosegaarde.net/project/dune/info/</footnote>
For an overview of similar ambient displays we refer the reader to Vande Moere [24]. What all of these ambient displays have in common is that they display information in an unobtrusive, yet aesthetically pleasing and fun way.

## 2.2 Persuasive ambient technology

Recent evidence suggests that ambient technology can also be employed for persuasive purposes (for example Breakaway [14] and Twinkly Lights [23]). The fact that such ambient systems are integrated into the environment, means they do not explicitly tell people to change their behavior. Instead they can suggest alternatives for a certain behavior [23], or provide implicit visualizations of 'bad' behaviors [14]. The strength of this approach lies in the notion that people are not consciously aware of the ambient intervention, yet still adjust their behavior [23].

There has been a substantial amount of research investigating the effects of static persuasive cues on the way people navigate public spaces. Such studies typically use textual [2] or visual cues [23] to guide people in a certain direction, for instance towards the stairs instead of the escalator. Examples of visual cues include lines that, when viewed from a certain angle, form guiding arrows [1]. Others used lenticular lenses to produce the illusion of lines moving to one side with the goal of guiding pedestrians [8].

Compared to the use of static visuals, the advantage of using ambient technology for persuasive purposes is that the system can be dynamically adapted to the users' behavior and to individual differences, as Kaptein says in [16]. This makes it more suitable for individual users, as well as allowing the system to adapt based on situational circumstances. For example a suddenly appearing obstacle that requires changing the direction in which pedestrians are guided. Here the use of smart materials is promising, because they offer both the ability to change dynamically as well as the ability to blend into the environment.

## 2.3 Application of smart materials

Considerable efforts have been made to explore the possibilities of applying smart materials in interfaces. For example, Surflex [3] consists of a foam with coiled nitinol muscle wires embedded. The muscle wires can be used to reshape the foam. This principle has been applied as a way to use SpeakCup [6], a device used to record and replay messages by physically manipulating its shape (Shape-changing interfaces [6]). Another suggested application of Surflex is as a form of physical computer-aided design. Indeed, Parkes and Ishii [19] presented just such a design tool, for motion prototyping and form finding, named Bosu. Bosu consists of a number of flexible modules that can be physically manipulated (e.g. bent, twisted). The physical manipulations can then be played back thanks to nitinol muscle wires embedded in the modules.

Apart from design tools such as Bosu, smart materials have been used in the creation of novel ambient displays. Shutters [5] is a curtain with a grid of shutters that are actuated by nitinol wires. Shutters can be used to regulate the amount of light and air flow that enters a room, as well as serving as an ambient display
by creating patterns using the grid of shutters. An ambient display that aims for
a more tangible experience is Lumen [20]. A grid of cylindrical physical pixels are
actuated to move up and down, using nitinol wires. The physical pixels contain
LED's that allow Lumen to display animations both visually, and physically.

While nitinol is by far the most widely applied smart material, others have used ferro fluids and thermo-chromic materials to create physical ambient interfaces. WeMe [17] for example, is an ambient display that visualizes the presence of members of remote families represented by bubbles of ferro fluid. Wakita et al. [25] note how, in the design of such interfaces, the materials and colors are felt as part of the emotional communication of the interface. Using the property (in this case colors) of the material directly, their aim is to communicate mood by changing the color of the Jello Display-Keyboard using IR-chromic material.

The application of smart materials in interfaces enables designers to build interfaces that can take advantage of human tactile senses and perceptions of movement to improve interaction [15]. This allows for the complete abandonment of the digital display in favor of shapes and movements, creating a more direct way to communicate with the user and establishing a novel design language that could reduce the cognitive load (unlike WIMP interfaces which add new symbols interpretation).

In the next section we will present our idea of applying smart materials in an ambient pervasive display that can be used for entertainment, indoor way-finding and ambient persuasive guidance. The keys to communication here are the shape and movements of the artificial blades of grass. As was mentioned in [18] we want to try to keep the perception of the interface as analog as possible, experimenting with materials and shapes.

3 Follow the Grass

Follow the Grass is an interactive pervasive display made of artificial blades of grass. By moving individual blades of grass sequentially, it is possible to create animations (e.g. a Mexican wave-style animation<fnref n="2" />). Such animations can be used to communicate information to users. While the grass-pixels were designed for general use as an ambient display, their appearance and movement capabilities, make them particularly well suited for giving directions in indoor way-finding, or for ambient persuasive guidance and entertainment.

Follow the Grass mimics the way grass waves in the wind, not by representing
only a single pixel but a mass of objects. In a large field installation it would be
possible to emulate a virtual wind by moving the grass in waving patterns. Such
a 'virtual wind' blowing through the grass field can be initiated by the presence
of the user (see chapters Scenarios and following). Using a dedicated tracking
system that follows and records the identity of users, it is possible to distinguish
between different users, and to address them individually.

<footnote n="2">² Mexican wave-style: it is a large scale example of a metachronal wave. These move-
ments produce the appearance of a travelling wave.</footnote>
The grass-pixels were born out of the idea that, by using smart materials, we can create interaction in a less traditional manner (no WIMP) and convey information through the materials' properties (shape, position, colors, etc.). By using a muscle wire<fnref n="3" /> we can accomplish a variety of movements allowing the blade to bend toward a specific direction (for now four cardinal point plus the four diagonals), with different speed to accentuate certain movement just by increasing the reaction time. Furthermore nitinol wires are highly reliable and resistant to many kinds of stress. Compared to servo-motors, the nitinol wires operate silently, not drawing the users' attention to the hardware operating behind the scenes. This allows a more seamless experience of the grass, making it ideal for pervasive applications.

Some previous works involved display methods similar to the one used in the Follow the Grass concept. The Project Dune has fibers with LED's embedded in the tips, that respond to sound. However, these blades of grass are not actuated and therefore lack the capability to address a certain direction. In Super Cilia Skin [22] the set of cilia creates a visual approximation of a contiguous deformable surface. Differently, the Follow the Grass concept attempts to make use of individually actuated blades of grass to address direction, or create a path by orienting individual blades of grass in unison. Compared to Coelho's work Sprout I/O [4] that uses nitinol wires to bend a fabric grass in a linear movement (left-right only), we make active use of two degrees of freedom (see next sections for more details of implementation and movements). Sprout I/O was born as a remote control device and interactive actuated interface. The Follow the Grass concept, on the other hand, was conceived instead as an actuated pervasive display, envisioned to be mass produced and to be put in different public spaces. In addition Follow the Grass should provide interaction designers with a flexible infrastructure to develop animations to be depicted in the grass.

Unlike real grass, on which you can walk, it is not possible to step on the artificial grass at this stage, however, in the future we will be able to build a more robust base which would allow users to walk on the grass. But a limited number of blades can be put on a pedestal, or, in the case of a grass field, under a transparent floor, allowing the user to walk over the grass still perceiving its presence and animations.

We envision the interaction to take place in public spaces where the users are free to walk such as a hall of a complex building, a crossing of corridors or just the entrance of a building. The tracking system handles the tracking of all individuals that pass the installation, and adds a special profile to each one of them. For example if you are in a hospital, and you need to know where the office of Dr. Red is, you can register your request at the reception desk. At this point the system can track your movements, and attach your request to you as an individual. By animating the grass in the direction of the office of Dr. Red, the only thing left to do is follow the grass distributed along the way.

<footnote n="3">³ Flexinol or muscle wire is special kind of nitinol wire that change shape by contraction its length.</footnote>
# 4 Scenarios

For the application of our system we illustrate several kinds of scenarios. In each of the scenarios the hardware and software will be the same, but in some cases adjustments are needed to adapt to the specific interaction. In every scenario the individual roots and blades of grass are the same. In some scenarios we assume that we know the objective of the user (such as: path or game objective), for example an existing “Helpdesk” or a pre-filled user profile.

## 4.1 Field

<figure><img src="image_2.png" /><figcaption>Fig. 1. Scenario Field. The users are walking on a transparent floor over the grass.</figcaption></figure>

In this scenario (Fig.1 and 2) we imagine a large-scale public installation, where a field of artificial grass is placed underneath a transparent floor. The users walk over the field without touching it, while the system tracks the identities of the users and produces personalized animation patterns under each user, by moving the nearby blades of grass. This can be used purely for entertainment purposes by simply displaying animations. For example a rippling effect, similar to a drop of water being dropped onto a water surface, causing concentric waves to form around the user. Another example is simulated wind that causes blades of grass all over the field to move in a natural manner. The field can also be used to guide individual users in a certain direction, by bending the blades of grass to form a path. A similar approach can be used to draw attention to specific objects in the space, by bending the blades of grass towards the object. Such an approach could be relevant for museum exhibitions or serious gaming purposes.

## 4.2 Lawn

In the “Lawn” scenario (Fig.3) the total number of blades of grass is reduced by placing strips of grass (lawns) along the walls of a public space. This ap-
<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Scenario Field. The grass bends creating a path from under the user to his/her goal.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Scenario Lawn. On the top the grass bends toward the door on the right side and toward the corridor on the left. On the bottom the detail of the grass bending toward the door.</figcaption></figure>
proach preserves part of the entertainment possibilities of the “Field” scenario and maintains the possibility to provide personalized guidance to users. Users could simply follow the direction of the waving grass along the wall. Attention can be drawn to specific areas along the wall, for example the office of the person the user is looking for.

## 4.3 Map

<figure><img src="image_5.png" /><figcaption>Fig. 4. Scenario Map. Installation from the side.</figcaption></figure>

Here, a specific location is addressed using a scale model of an area (Fig.4 and 5), for example a university campus. The scale model will represent the surrounding area, with the space between the models covered in the artificial grass. Users can interact with the scale models through gestures (using the Kinect included in the system) in order to ask for directions. The grass will respond by forming a path from the users current location to the requested destination. This setup can also be used to highlight events in a specific part of the area, by playing animations that focus on the area in which the event takes place.

## 4.4 Patch

The last scenario (Fig.6) is a small scale field (3x3 blades of grass) coupled with the tracking system. The patch of artificial grass can be placed on a pedestal at the place where a corridor splits. In the presence of the user, the grass will attract the users attention in *entertainment mode* (waving in different pattern or bending toward the user) and if profiled, address a direction in *guiding mode*.
<figure><img src="image_6.png" /><figcaption>Fig. 5. Scenario Map. The grass is bending toward the tower.</figcaption></figure>

# 5 System: tracking and actuation

The system is composed of two distinct components: one for the physical output (the grass) and one for the input, which is the presence of the users detected by the tracking software.

With the system, it is possible to track a specific person in a small group of people. This makes it possible to address this specific person individually, and provide him/her with personalized information, such as specific directions at the right time.

The interaction starts when people enter the area covered by the tracking system.

## 5.1 Software

A Kinect<fnref n="4" />-based tracking system is being developed to be able to address and track all the users in its sight. The idea is to have a system that is able to track identities from a simple blob tracker<fnref n="5" />. We intend to identify the user by knowing the tracking faults of the blob tracker itself and solving the problem in two phases. First, each time a blob merges with another we will trace and

<footnote n="4">⁴ http://www.microsoft.com/uk/wave/hardware-kinect.aspx</footnote>

<footnote n="5">⁵ With BLOB we refer to regions with no fixed shape in the image that differ in properties compared to the surrounding. E.g.: brightness or color.</footnote>
<figure><img src="image_7.png" /><figcaption>Fig. 6. Scenario Patch. The grass bends indicating a direction to the incoming user.</figcaption></figure>

save the previous and new relation (and related information), creating a sort
of logical history of relation of blobs (with “AND” and “OR” logic conditions).
Second, we will try to confirm the identity (between the observed ones) using
logical operation and matching with hints that the software acquires from the
behavior and features analysis of each blob (in development).

The user can be identified in one of the specific checkpoints (if needed for
giving personalized information) or just by entering the sight zone, a numeric
ID is attached to the blob and tracked during the movements.

When the system is unable to track the user, there are three possibilities:

- The user is out of sight.

- The blob has merged with someone else's blob. In this situation we aggregate the information in "AND" until the system is able to address the blobs again.

- The blob has split into two or more parts. The information of the old blob is inherited in a doubt condition, i.e. an "OR" mode, on the generated blobs.

This way we generate blobs with some of the information precisely (aggregation
of reliable information in “AND”) and others in doubt (separation of blobs,
in “OR” condition). To solve the “OR” condition we use a feedback loop that
checks the features of the new blobs from time to time (shape, contrast, color,
height, etc.) with the one from before. Basically giving hints on which one is the
most reliable solution. We then backward propagate on the other blobs to solve
other pending doubt conditions.

The system has some limitations and can only be used in a controlled envi-
ronment (at the moment only inside the field of view of the Kinect). However, it
is modular so that it will be further developed with other tracking features and
the possibility to extend the field of view by connecting more units together.

5.2 Physical

Each grass-pixel is made of three parts: the controlling electronics, the “root”,
or actuator base (Fig.7), and the externally visible blade of grass. The electronic

<figure><img src="image_8.png" /><figcaption>**Fig. 7.** Physical actuator element, It is possible to see the movable head of the “root”, where the blade of grass will be attached. A) the original 3D model for the project, B) the cardboard prototype with spring, C) latest prototype update with silicon“spring”.</figcaption></figure>

component consists of an Arduino<sup>6</sup> board that controls the power on a specific duty cycle to the muscle wire, using a PWM (Pulse-Width Modulation). This way it is possible to control the amount of power used to heat the wire. It is possible to vary the duty cycle to make the heating process faster or slower, resulting in faster or slower movement of the grass, allowing for controlled organic movements and animations.

The root is the physical actuator that forms the base on which the blade of grass is placed. The root is hidden from view, leaving only the blade of grass

<footnote n="6">6 Arduino is an open hardware platform http://Arduino.cc, one board controls one blade of grass, but with some optimization, the number of blades can be incremented.</footnote>
visible. Originally made of a long cardboard prism with squared base, it is now
substituted with a more practical wood splint. The base measures 20 centimeters
in height, to allow enough contraction space for the muscle wire. The wires
contract longitudinally about 4% of their total length. Attached to the main
structure is a spring that holds a square that serves as the platform on which
the blades of grass are attached. At the end of each edge of the top, a nitinol
muscle wire is attached. The other end of the wire is connected to the lowest
part of the main structure. The wires were originally kept in tension by a spring,
but this created a balancing problem. The latest version has a soft silicon mold
for enhanced stability.

The blade of grass will be affixed to the top part of the root, and will be
made out of a flexible material so that when the base moves, the blade of grass
gently sways.

The reaction time of the actuator is relatively fast, taking about half a second
to contract. Here we refer to the complete contraction time, from resting position
to the end of the animation. In order to avoid a "switch-click" effect (sudden
movement from initial position to the final), we slowed down the reaction to
a more "organic" contraction speed of around one second. Combined with a
relaxation time of around one-and-a-half seconds<fnref n="7" />, the animation of a waving
piece of grass is nearly symmetrical.

**6 Conclusion and future works**

Our aim in this paper was to describe the ongoing work in building an interac-
tive physical pervasive display, that can be used for indoor way-finding, ambient
persuasive guidance, as well as for entertainment. Following the visions of ubiq-
uitous computing and tangible interfaces, the Follow the Grass concept offers
an engaging way for users to interact with their environment. As it has been
demonstrated in [23] and [8] it is possible to change the behavior of people by
giving peripheral visual cues. Follow the Grass has the potential to have the same
effect. Furthermore, the tracking system enables the interface to react dynam-
ically to the users' presence, making it possible to give the correct information
at the right time to the correct individual. This creates a more personalized ex-
perience for the user. The capability of the artificial grass to display animations
adds to the expressiveness of the interface, and creates opportunities to engage
and entertain the user.

The design of the actuated root is still ongoing. We are working to improve
on a number of issues, including the precision of the addressed direction, as well
as more secure and unobtrusive attachment of the nitinol muscle wires to the
base. Once we have a sufficiently robust version of the actuated root we aim to
realize the ‘patch’ scenario consisting of a grid of 3x3 blades of grass. Future
developments for the actuated root could include installing a tilt sensor on top
which would reduce the number of nitinol wires required, from four to three.

<footnote n="7">⁷ This amount of time is required for relaxing without forcing the wire.</footnote>
The sensor would enable measurement of the angle of the platform to which
the blade of grass is attached, allowing fine adjustments to be made using only
three muscle wires. The next step would be to make a serializable version of
the root for creating a larger field of artificial grass. Miniaturizing the electronic
components and constructing a dedicated controller module, would make the
system more suitable for modular deployment. The tracking software will have
to be improved and optimized for larger scale interactions (e.g. big rooms and
halls). We are in contact for a possible future version of “Lawn” in a museum.

Acknowledgements

This publication was supported by the Dutch national program COMMIT and
a CTIT-ITC collaboration project.

References

1. Boehm, G.: Ambient persuasive guidance. In: Proc. of the 5th international conference on Tangible, embedded, and embodied interaction. TEI '11, New York, NY, USA, ACM (2011) 431–432

2. Brownell, K., Stunkard, A., Albaum, J.: Evaluation and modification of exercise patterns in the natural environment. American Journal of Psychiatry (1980)

3. Coelho, M., Ishii, H., Maes, P.: Surflex: a programmable surface for the design of tangible interfaces. In: CHI '08 ext. abstracts on Human factors in computing systems. CHI EA '08, New York, NY, USA, ACM (2008) 3429–3434

4. Coelho, M., Maes, P.: Sprout I/O: a texturally rich interface. In: Proc. of the 2nd international conference on Tangible and embedded interaction. TEI '08, New York, NY, USA, ACM (2008) 221–222

5. Coelho, M., Maes, P.: Shutters: a permeable surface for environmental control and communication. In: Proc. of the 3rd International Conference on Tangible and Embedded Interaction. TEI '09, New York, NY, USA, ACM (2009) 13–18

6. Coelho, M., Zigelbaum, J.: Shape-changing interfaces. Personal Ubiquitous Computing **15** (February 2011) 161–173

7. Fitzmaurice, G.W., Ishii, H., Buxton, W.A.S.: Bricks: laying the foundations for graspable user interfaces. In: Proc. of the SIGCHI conference on Human factors in computing systems. CHI '95, New York, NY, USA, ACM Press/Addison-Wesley Publishing Co. (1995) 442–449

8. Furukawa, M., Yoshikawa, H., Hachisu, T., Fukushima, S., Kajimoto, H.: “Vec-
tion field” for pedestrian traffic control. In: Proc. of the 2nd Augmented Human
International Conference. AH ’11, NY, USA, ACM (2011) 19:1–19:8

9. Heiner, J.M., Hudson, S.E., Tanaka, K.: The information percolator: ambient information display in a decorative object. In: Proc. of the 12th annual ACM symposium on User interface software and technology. UIST ’99, New York, NY, USA, ACM (1999) 141–148

10. Ishii, H., Lakatos, D., Bonanni, L., Labrune, J.B.: Radical atoms: beyond tangible bits, toward transformable materials. interactions **19**(1) (January 2012) 38–51

11. Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proc. of the SIGCHI conference on Human factors in computing systems. CHI '97, New York, NY, USA, ACM (1997) 234–241
12. Ishii, H., Wisneski, C., Brave, S., Dahley, A., Gorbet, M., Ullmer, B., Yarin, P.: ambientroom: integrating ambient media with architectural space. In: CHI '98 conference summary on Human factors in computing systems. CHI '98, New York, NY, USA, ACM (1998) 173–174

13. Jacob, R.J., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., Zigelbaum, J.: Reality-based interaction: a framework for post-wimp interfaces. In: Proc. of the 26th annual SIGCHI conference on Human factors in computing systems. CHI '08, New York, NY, USA, ACM (2008) 201–210

14. Jafarinaimi, N., Forlizzi, J., Hurst, A., Zimmerman, J.: Breakaway: an ambient display designed to change human behavior. In: CHI '05 ext. abs. on Human factors in computing systems. CHI EA '05, New York, NY, USA, ACM (2005) 1945–1948

15. Jung, H., Altieri, Y.L., Bardzell, J.: Skin: designing aesthetic interactive surfaces. In: Proc. of the fourth conf. on Tangible, embedded, and embodied interaction. TEI '10, NY, USA, ACM (2010) 85–92

16. Kaptein, M., Markopoulos, P., de Ruyter, B., Aarts, E.: Persuasion in ambient intelligence. Journal of Ambient Intelligence and Humanized Computing 1 (2010) 43–56 10.1007/s12652-009-0005-3.

17. Masson, N., Mackay, W.E.: Weme: Seamless active and passive liquid communication. In: Proc. of the 13th International Conference on Human-Computer Interaction. Part II: Novel Interaction Methods and Techniques, Berlin, Heidelberg, Springer-Verlag (2009) 694–700

18. Minuto, A., Vyas, D., Poelman, W., Nijholt, A.: Smart material interfaces: A vision. In: Proc. 4th International ICST Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN 2011). LNICST 78, Springer (2011) 57–62

19. Parkes, A., Ishii, H.: Bosu: a physical programmable design tool for transformability with soft mechanics. In: Proc. of the 8th ACM Conference on Designing Interactive Systems. DIS '10, New York, NY, USA, ACM (2010) 189–198

20. Poupyrev, I., Nashida, T., Maruyama, S., Rekimoto, J., Yamaji, Y.: Lumen: interactive visual and shape display for calm computing. In: ACM SIGGRAPH 2004 Emerging technologies. SIGGRAPH '04, New York, NY, USA, ACM (2004) 17

21. Prante, T., Stenzel, R., Röcker, C., Streitz, N., Magerkurth, C.: Ambient agoras: Inforiver, siam, hello墙. In: CHI '04 ext. abstracts on Human factors in computing systems. CHI EA '04, New York, NY, USA, ACM (2004) 763–764

22. Raffle, H., Joachim, M.W., Tichenor, J.: Super cilia skin: An interactive membrane. Human Factors (2003) 808–809

23. Rogers, Y., Hazlewood, W.R., Marshall, P., Dalton, N., Hertrich, S.: Ambient influence: can twinkly lights lure and abstract representations trigger behavioral change? In: Proc. of the 12th ACM international conference on Ubiquitous computing. Ubicomp '10, New York, NY, USA, ACM (2010) 261–270

24. Vande Moere, A.: Beyond the tyranny of the pixel: Exploring the physicality of information visualization. In: Information Visualisation, 2008. IV '08. 12th International Conference. (july 2008) 469–474

25. Wakita, A., Shibutani, M., Tsuji, K.: Emotional smart materials. In Jacko, J., ed.: Human-Computer Interaction. Ambient, Ubiquitous and Intelligent Interaction. Volume 5612 of LNCS. Springer Berlin / Heidelberg (2009) 802–805

26. Weiser, M.: The computer for the 21st century. Scientific American (1991) 94–104

27. Wisneski, C., Ishii, H., Dahley, A., Gorbet, M., Brave, S., Ullmer, B., Yarin, P.: Ambient displays: Turning architectural space into an interface between people and digital information. In: Cooperative Buildings: Integrating Information, Organization, and Architecture. Volume 1370 of LNCS. Springer Berlin / Heidelberg (1998) 22–32 10.1007/3-540-69706-3_4.