<figure><img src="image_1.png" /></figure>

HAL
open science

# Interactive Environments: A Multi-disciplinary Approach towards Developing Real-Time Performative Spaces

Nimish Biloria

► To cite this version:

Nimish Biloria. Interactive Environments: A Multi-disciplinary Approach towards Developing Real-Time Performative Spaces. 9th International Conference on Entertainment Computing (ICEC), Sep 2010, Seoul, South Korea. pp.254-261, 10.1007/978-3-642-15399-0_25 . hal-01055637

HAL Id: hal-01055637

https://inria.hal.science/hal-01055637

Submitted on 13 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Interactive Environments: A multi-disciplinary approach towards developing real-time performative spaces

Nimish Biloria

Hyperbody, Faculty of Architecture, TU Delft, Julianalaan 134
2628 BL Delft, The Netherlands, N.M. Biloria@tudelft.nl

**Abstract.** The research paper exemplifies upon a series of real-time information exchange driven design-research experiments conducted by the Hyperbody research group (HRG), Faculty of Architecture, TU Delft. These interactive spatial prototypes, while successfully integrating the digital with the physical domains, foster multiple usability of space and are appropriately termed as ‘The Muscle Projects’ based on the pneumatic muscle driven actuation technologies used per project. The interactive nature of the projects is realized through harnessing a synergistic merger between the fields of ambient sensing, control systems, architectural design, pneumatic systems and computation (real-time game design techniques). The prototypes are thus visualized as complex adaptive systems, continually engaged in activities of data-exchange and optimal augmentation of their morphologies in accordance with contextual variations.

**Keywords:** Interaction design, Multi-disciplinary, Real-time data exchange, Control systems, Dynamic structures

## 1. Underpinning

The design-research experiments developed at Hyperbody, TU Delft, Faculty of Architecture focus on the emerging inter-disciplinary field of Interactive Architecture focusing on developing real-time information exchanging architectural spaces. These interactive spaces demonstrate a fusion between the material, the electronic and the digital domains. This fusion is explicitly attained through harnessing a synergistic merger between the fields of ambient sensing, control systems, ubiquitous computing, architectural design, pneumatic systems and computation (real-time game design techniques). The resultant spaces, visualized as complex adaptive systems, are continually engaged in activities of data-exchange and interact via physical and ambient adaptations of their morphologies in response to sensed contextual variations.

A strategic co-evolution of technical knowledge between the industry (specifically Festo, a pneumatic engineering company), praxis (ONL a multi-disciplinary design firm, The Netherlands), and academic research (Hyperbody for spatial and information design) gives shape to these interactive constructs, hence
developing an information bridge between these three critical knowledge sectors. The collaboration process involved a series of associative brain storming, simulation and prototype testing sessions focusing upon the usage of pneumatic and electro pneumatic technologies, interaction design concepts, deciding upon appropriate control systems, structural stability and performance aspects concerning the conceptualized spatial configuration of the system. Experimentations with material systems as regards their flexibility, shape retention and strength ratios coupled with kinetic structural systems formulated a vital part of the collaborative effort. Parallel research and development in interaction design, ubiquitous communication, creating computational routines via software such as Virtools and Max MSP as well as developing project specific sensing networks were also carried out under these collaborative design initiatives.

A systems thinking approach with an emphasis on ‘Interdependence and Interconnection’ between variable contextual parameters (global) as well as the components formulating a prototype (local) is embarked upon as a part of the synergistic merger of the aforementioned fields. This implied that even the smallest detail in any prototype, for instance every node/joint is conceived as a potential information hub and has the ability to collect, process and communicate contextual data. Apart from this, the same node/joint, via mechanical actuators, is able to kinetically re-position itself physically in three-dimensional space, thus enabling local structural adaptations, which cumulatively result in complex morphological variation.

## 2 System components

Issues such as multiple usability of space via spatial augmentation (at a functional level), ambient augmentation (for suggesting the mood/emotional state of the space) as well as information augmentation (as regards the occupancy as well as attraction of the space) attained paramount importance for creating the real-time interactive prototypes. A detailed list of the system components, further segregating them in accordance with their generic usage led to the identification of the following:

a. Pneumatic Entities: Fluidic Muscle Type MAS (provided by Festo): A flexible tube with reinforcing fibers in the form of a lattice structure for up to 10x higher initial force compared to a cylinder of identical diameter. The muscles (Fig 1) tend to contract 20 percent of their initial length with the induction of air pressure, hence making it act as an actuating device to alter the node positions of the prototype.

b. The Black Box (by ONL and HRG with Festo air valves and switching components): A hard-edged box housing the switching mechanisms: I/O boards connected to the 72 valves controlling the air pressure lock of the Fluidic muscles. The box has provisions to attach the compressed air intake pipes through distribution channels; houses the CPU and power back up mechanisms (Fig 1). At a later stage the Black Box was replaced by Festo's CPX controllers (Fig 1).

c. Flexible Skins (utilized in different prototypes): Hylite panel, a sandwich sheet comprising two thin aluminum layers with a plastic core in between (Fig 1) which integrates high flexural stiffness and extreme lightness. Lycra based stretchable fabric
(Fig 1) normally used for sports clothing whose translucency varies according to the degree of stretching is also used within some prototypes.

d. Control System: Proximity sensors for sensing the distance of the occupant from the installation, Touch sensors for sensing the amount of pressure exerted upon a surface, Tilt sensors for determining the x,y,z axis displacements. MIDI, Digital to Analogue and vice versa converters: connecting the sensor input channels and the actuator output channels to and from the CPU. PCI cards, are programmed to receive the output signals (a long numeric string corresponding with the number of fluidic muscles, indicating the status shift of each node: on/off) and communicate them to the In/Out board mentioned earlier (black box), hence controlling the airlock valves.

e. Real-time Game-Design Software: Virtools Dev 3.0, software is used for developing an inherent connectivity between the sensed data and the expected behavior output from the prototype (by means of programming output rules for the system). The software is used as the main computation tool which receives inputs from the MIDI device (sensed data), processes data in accordance with the scripted behaviors programmed into it and sends output digital signals via PCI cards device, which are directly linked with actuating mechanisms.

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Components used in the prototypes: Top left to right, Fluidic Muscles, Black Box, CPX controller, I Cube MIDI digitizer, and the bottom row showing left to right the Hylite panels, Lycra skins and the Neoprine envelope.</figcaption></figure>

### 3. Prototypes

At Hyperbody, a series of prototypes to study the intricacies of research driven design of interactive architectures have been developed over the past six years. The first in the series called the NSA Muscle was specifically built for the NSA (Non Standard Architecture) exhibition in Centre Pompidou, Paris, in 2003 as a collaborative project between ONL, Hyperbody and Festo.
## 3.1 The NSA Muscle

The NSA Muscle is a pro-active inflated space, its surface populated with a mesh of 72 pneumatic muscles, which were all addressed individually by means of regulating the amount of air pressure induced within them. The prototype is programmed to respond to human visitors through its sensing, processing and actuating enhancements.

Design principles: To communicate with the observers, the NSA Muscle has to transduce physical quantities into digital signals (sensors) and vice versa (actuators). People connect to the NSA Muscle by 24 sensors attached to reference points on the structure. These input devices convert the behavior of the human players into data that acts as the parameters for changes in the physical shape of the active structure and the ambient soundscape. The input setup comprises eight sensor nodes (Fig 2) with three sensors each: motion (for sensing the presence of possible players from a distance of 6 meters), proximity (for sensing the distance of the players to the NSA Muscle within a distance of 2 meters) and touch (for sensing the amount of pressure applied upon the surface). The analogue sensor input channels are converted to digital audio signals (MIDI) and transferred to the computer.

The NSA Muscle is programmed to behave within predefined bandwidths of emotional modes including jumping (excited), retracting (scared) and shivering (anger) which are attained via tactile variations of its external form (by changing the length of the tensile muscles) accompanied by the emission of pre-defined sounds of variable pitch. A three-dimensional visualization (Fig 2) of the Muscle rendered on a computer screen informs the people about the state of this being. The activity of the muscles is displayed in three colors in the model: red / inflating state, blue / deflating state, and gray / passive state, and in the internally used organizational 72 digit string. Images of practical architectural applications, using muscle technology, complement the graphical display. The real time model is actively viewed from multiple camera positions so as to feel the behavioral patterns at work. Viewed in combination with the physical model this graphical interface contributes to the public's level of understanding.

User evaluation: The Muscle NSA owing to its emotional modes, proved to be an attraction for many users. The jumping mode, apart from being activated when many people were interacting with it, was also programmed to activate when the sensing field could not track people around it. This was treated as a strategy to seek attention of the passers by and invite them to interact with the Muscle. Once attracted, the users could also sense the shivering mode (accompanied by a low pitch sound) followed by the retraction of the nodes of the Muscle (at a local level) with which people were interacting. This behavior engaged people in a game play between architectural space and their own emotional selves, thus aiding in changing the perception of architectural space from a static object to a dynamic subject.
<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Left: Visitors interacting with the NSA Muscle sensor nodes at the Centre Pompidou, Paris, 2003, Right: The NSA Muscle’s real-time computational process being shown via a monitor at the NSA exhibition. The activity of the physical muscles is displayed in the digital world in real-time in three colors in the model: red / inflating state, blue / deflating state, and gray / passive state, and in the internally used organizational 72 digit string.</figcaption></figure>

## 3.2 The Muscle Re-Configured

The Muscle Re-Configured project was specifically aimed at materializing a real time responsive variant of the Muscle NSA project by ONL and HRG. The reconfiguration is realized by means of utilizing the same actuating components: pneumatic Festo fluidic muscles from the Muscle NSA project. However for the new installation, instead of the soft volumetric alterations of the external form (as was materialized through the Muscle NSA project), an approach emphasizing internal spatial response is visualized.

Design principles: In order to experiment with an interior space, the re-configured prototype was conceived as a 3D habitable Strip (Fig 3): a three dimensional section in space, programmed to respond to its occupants through its sensing (proximity and touch sensors), processing (graphical scripting for real-time output) and actuating (fluidic muscles) enhancements. The construct utilizes the properties of the Hylite panel's being bendable and the fluidic muscle's being able to produce compression power to interact upon each other, thus transforming the otherwise hard edged (visually) spatial strip into soft luxuriant variations. Each Hylite panel coupled with two fluidic muscles hence forms the basic unit of the strip. Subsequent panels are joined together to create a closed 3D loop, in the process creating a series of nodes (junctions where the panels join). These nodes, owing to the possession of actuation members, are linked in space in a highly interdependent manner, constantly exchanging information (in terms of air pressure variations), yet, behaving as a collective whole to attain spatial reconfigurations. The collective whole as such constitutes the following: the responsive floor, ceiling and walls:

The floor units (Fig 3) are strengthened by means of adding wedge shaped Styrofoam sections underneath the inner surface of Hylite panels for seating and relaxation purposes. They also embed touch/pressure sensors (manual control) which, when pressed produce curvature changes by means of varying the amount of air
pressure released to the fluidic muscles of the seats. The floor units also embody
proximity sensors at their base, which, instantly after sensing the proximity of people
trigger the seating units to a pre-programmed position, to create suggestive usability
scenarios. The positioning/orientation of the panels constituting the ceiling unit's (Fig
3) is directly related with their affordance (functional and psychological). The ceiling
units behavior involves the creation of projection surfaces, generation of smooth
curvilinear soothing forms for relaxation purposes and for materializing openings in
ceiling surface for allowing light to venture through. These behaviors are attained by
means of utilizing the compression power of the corresponding sets of fluidic muscles
attached to the Hylite sheet units. The ceiling's tactile variations specifically deal with
the users input, specifically the touch sensor readings obtained from the seating units;
The ceiling units directly above the seating units are bound together in a way that any
curvature variation in the seating units shall have a corresponding curvature variation
in the ceiling units. Pressure exerted on the touch sensors in the seating units,
therefore creates a subsequent change in configuration of the ceiling units. The wall
units (Fig 3) constitute the same generic Hylite panels, which are woven together to
create a continuous surface with the ceiling elements. The same principle of
compression strengths goes into materializing the wall, which, when actuated bends
to create projection surfaces and higher seating surfaces (Figure 3). The actuation of
the wall and ceiling elements are also intrinsically linked up with the seating element
actuations, hence weaving the entire construct into a cumulative whole. However
there are also provisions in which, for experimental reasons, one can individually
trigger the three entities.

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** The three spatial elements constituting the 3d loop of the Muscle Reconfigured; Floor unit's construction and usability scenarios, Ceiling units (displaying controlled opening behavior), Wall units bending to create projection surfaces.</figcaption></figure>

**3.3 Emotive InteractiveWall**

The Emotive InteractiveWall (Fig 4) is composed of 7 separate wall pieces (herein referred to as nodes) that display real time behavior by swinging back and forth, displaying patterns of light on its skin, and projecting localized sound.

Design principles: The primary synchronous behavior of the InteractiveWall is movement. The nodes of the InteractiveWall will bend independently of
neighboring nodes in response to the presence of a user. Although responsively
independent, each InteractiveWall node synchronizes by constantly readjusting its
position in order to align itself with the position of its nearest neighbors.

Augmented modality of the InteractiveWall's behavior is light. The skin of each InteractiveWall is covered by a unique, irregular distribution of dynamically controlled LEDs that form a highly reactive interface. The LED skins respond directly to user presence by glowing brighter when users are near, and dimmer as they move away. In addition to dimming, the LED skins pulse rapidly and slowly in relation to node position, having a tendency to flash together when the nodes are in sync.

The third modality of the InteractiveWall is localized sound. Moments of synchronicity are represented by calmer sounds, while asynchronous behavior results in more intense sound. The propagation of the sound from high to low intensity is varied throughout the InteractiveWall, thus each node is a member of a choir that sings a complex pattern of oscillating chords. Although similar, the physical movements of InteractiveWall, and the light and sound patterns change independently, reacting at varying rates. The synchronous behavior between the InteractiveWall nodes contrasts with the behavior produced by user presence, resulting in a series of complex wave patterns that propagate through the InteractiveWall structure as a whole.

User experience: The users interacting with the Emotive InteractiveWall experience the installation as an interactive system that exhibited emergent behavior and performed like a living system. The wall worked as an independent system built on synchronous behavior that is interrupted by the game-like response of multi-participant interaction. This layered system encourages the intended cycle of observation, exploration, modification, and reciprocal change in the participant, reinforcing believability in the system, and providing a sense of agency to the user.

<figure><img src="image_6.png" /><figcaption>**Fig. 5.** Top: The InteractiveWall at the Hannover Messe 2009 displaying real time behavior by swinging its body back and forth, displaying patterns of light on its skin, and projecting localized sound.</figcaption></figure>
## 4. CONCLUSION

An intuitive interaction, opinionated towards seamless information exchange is initiated through the research experiments, hence transforming everyday utilitarian space into an entertaining, inter-activating responsive organism. The prototypes fuel the idea of developing pro-active spaces communicating and reconfiguring in real-time, while being sensitive towards their context. The successful accomplishment of the projects is also suggestive of the benefits yielded by a collaborative manner of working with varying fields of expertise and stresses upon further envisioning emotive architectural beings which understand and respond to its occupants.

The aforementioned techniques of developing interdependent nodal networks, which double up as actuated details of interactive bodies, stress upon scripting localized relations between constituent details, thus actualizing the performance of the built form as an emergent, communicative exchange between the network of components. Information flow becomes a continual process in such real-time interactive prototypes, converting them into executable processing and reacting systemic entities. Such architectural constructs eventually acquire the characteristics of living entities, sending and receiving information, processing this information locally, and producing optimal global output.

The intrinsic design decision of enriching the nature of architectural detailing and establishing an inter-disciplinary work process has significant impact on the nature of architectural space and its structuring principles. Such design-informatics-based hybrid typologies can be seen as highly logical systems, which, while being entertaining due to its shear grandeur and kinetic abilities, pave the path for performative responses to contemporary contextual complexities. These research experiments thus initiate an intuitive interaction, inclined towards seamless information exchange, transforming everyday utilitarian space into an inter-activating responsive experience.

## References

1. Mc Luhan: Understanding Media. Routledge Classics, London and New York (2002)

2. Lupton E.: Skin, Surface Substance + Design. Laurence King Publishing.Ltd, London (2002)

3. Derrick de Kerckhove: The architecture of intelligence, Basel, Birkhauser (2001)

4. Srinivasan, A.V.. Michael McFarland, D.: Smart structures analysis and design, Cambridge UK, Cambridge University Press (2001)

5. Reynolds C.W: Flocks herds and schools: A distributed behavioral model, Computer graphics, 21, SIGGRAPH’87, conference proceedings.

6. Bossomaier, TRJ and Green DG: Complex systems, Cambridge university press, UK (2000)

7. Holland JH: Emergence: from chaos to order, Oxford University Press, London (1998)

8. Kerkhove, D: The architecture of intelligence, Birkhauser publication, Basel (2001)

9. Oosterhuis K: Architecture goes wild, 010 publications, Rotterdam, The Netherlands (2002)

10. Oosterhuis K: Hyperbodies: Towards an E-motive Architecture, Birkhauser, Basel (2003)

11. Emile Aarts and Stefano Marzano (Ed): The new everyday: views on ambient intelligence, 010 publishers, Rotterdam, The Netherlands (2003)