# Linetic: Technical, Usability and Aesthetic Implications of a Ferrofluid-Based Organic User Interface

Jeffrey Valino Koh, Kasun Karunanayaka, Ryohei Nakatsu

► To cite this version:

Jeffrey Valino Koh, Kasun Karunanayaka, Ryohei Nakatsu. Linetic: Technical, Usability and Aesthetic Implications of a Ferrofluid-Based Organic User Interface. 14th International Conference on Human-Computer Interaction (INTERACT), Sep 2013, Cape Town, South Africa. pp.180-195, 10.1007/978-3-642-40477-1_11. hal-01504883

HAL Id: hal-01504883

https://inria.hal.science/hal-01504883

Submitted on 10 Apr 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Linetic: Technical, Usability and Aesthetic Implications of a Ferrofluid-based Organic User Interface

Jeffrey Tzu Kwan Valino Koh, Kasun Karunanayaka, Ryohei Nakatsu

Keio NUS CUTE Center, Interactive and Digital Media Institute, National University of Singapore, Singapore

jtkvkoh@gmail.com, kasun@mixedrealitylab.org, elenr@nus.edu.sg

**Abstract.** We present an OUI that combines Hall Effect sensing and actuation through electromagnetically-manipulated ferrofluid. The movement of magnets worn on the fingertips, over a surface embedded with a Hall Effect sensor array and electromagnets, gives the user the ability to interact with ferrofluid. This system provides a three-dimensional, physically animated response, as well as three-dimensional, spatial-sensing inputs. The vibration of the magnets worn on the fingertips, produced by the repulsing polarity of the electromagnets, provides the user with haptic feedback. Linetic is a multimodal interface with a visual, audio and haptic experience. In this manuscript we explain the overall system from a technical, usability and aesthetic viewpoint by outlining significant experiments conducted that contribute to the development of the system. Furthermore we discuss the philosophical and aesthetical implications of the Linetic system, as well as characterize Linetic's disposition to Analogness or Digitalness.

**Keywords:** Electromagnetic; haptic; TUI; OUI; morphable; shape changing.

## 1 Introduction

Recent progress in human-computer interaction has paved the way for a new body of research known as Organic User Interfaces (OUIs) [2] [10]. This field focuses on the need to further explore possibilities for interactive user interfaces using the advances made in electronics and material sciences [9]. OUIs are defined by three factors: the input interface and output display should be one and the same; the form of the object should change continuously and correlate directly with the function it embodies, and finally the function performed by the object depends on how the physical shape of said object is changed [22].

OUIs open a path to morph the shape and form of the actual computer interface itself. The interface can now be a piece of fabric, a plastic card, liquid, sand, clay or any other material. OUIs enable users to interact with the interface by manipulating the natural qualities of these materials such as bending, stretching, pulling, stroking, etc. This new paradigm moves away from the traditional approach of metaphors and
physical objects as defined by Tangible User Interfaces (TUIs), and explores next generation interfaces focused on the analog, continuous and transitional nature of physical reality and human experiences, as Schwesig points out [20].

With advances in technology pushing the boundaries in regards to the materials used to create ubiquitous interactive systems [24], it is now possible to expand the computer into the everyday environment through softer, and flexible formats [1]. Using these materials and technologies, this paper presents the implementation of an innovative OUI system based on liquid. This interface explores the potential of liquids as an interface and display device, where the manipulation of liquid becomes both the input and output. Linetic (Link to youtube video - http://www.youtube.com/watch?v=V5EMyRsLLeQ) can provide the user with a natural and fluid experience where three-dimensional, tangible interaction takes place.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Version one of the Linetic system.</figcaption></figure>

Building on the idea of a ferrofluid display created by Sachiko Kodama [13], Linetic provides an input/output solution based on ferrofluid. Ferrofluid is essentially a liquid that reacts to magnetic fields. The system is composed of a pool of ferromagnetic liquid combined with a sensing and actuation mechanism. The sensing uses an array of Hall Effect sensors that measure the density of a magnetic field, while actuation is produced by an array of electromagnets. Sound generation using a MAX/MSP patch running on a connected server augments the output experience.

Wearing a set of magnetic rings, the user can interact with the ferrofluid. The magnetic ring position is detected by the array of Hall Effect sensors, which in turn actuates the electromagnets and the sound server. The magnetic field of the active electromagnets produces the morphing of the ferrofluid to create transitional physical buttons in conjunction with the gesture, which then generates a sound. At the same time the pulse of the matching polarity electromagnets produces a force feedback vibration on the rings, giving the user haptic feedback.

Through natural movements of the hand, the interface is able to morph from a two-dimensional surface to a three-dimensional form fluidly and dynamically. Using the shape changing quality of ferrofluids, we were able to study how liquids could be-
come a novel form of OUI. In sum Linetic provide a tangible, multi-touch interface with haptic feedback that produces a real 3D morphing surface.

## 2 Background

Linetic builds on research by both scientists and artists in creating a malleable, three-dimensional, multi-touch interface. Hiroshi Ishii's seminal research in TUIs offers insight for natural user interfaces in which this project builds upon [9]. Ishii later went on to categorize generations of TUIs, leading to the development of guidelines for the next iteration of user interfaces, known as Organic User Interfaces [8] [10] [23].

Research into the field of OUIs is still in its infancy. In 2008 ACM published a groundbreaking collection of essays that collected the works of OUI's pioneering researchers in a special issue of Communications of the ACM. In this special issue, several key authors outlined various aspects and characteristics that make an OUI:

*   **Input Equals Output:** In the graphical user interface (GUI) there is a clear division of input and output. The mouse and keyboard input actions from the user. Based on those actions, output is generated graphically on the screen. A key feature of OUIs is that a piece of Organic light-emitting diode paper, or any potentially non-planar object for that matter, is meant to input actions from the user and also outputs them onto the same object.

*   **Function Equals Form:** The form of an object clearly determines its ability to be used as an input. The statement Function Equals Form emphasizes this dependency on one another. Holman and Vertegaal [8] argue that these two are in fact inseparable and that it is a mistake to try to deny this in any way.

*   **Form Follows Flow:** This principle states that it is of utmost necessity for OUIs to negotiate user actions based on context, e.g. the ubiquitous 'clamshell' phone, where incoming calls alter the phone's function when opening the phone during an incoming call.

In regards to the material we choose to use for our input and output, the ferromagnetic art installations by Sachiko Kodama provide an aesthetic viewpoint of how the power of fluid and transient shapes can capture the imagination of viewers [13]. In her projects ferromagnetic fluids are used to create organic shapes that change structure dynamically. Adjusting the power of the electromagnets actuates the ferromagnetic fluid in varying ways. The magnetic field produced by the electromagnets controls the movement of the ferrofluid, producing a visual output. Linetic builds on this work, providing the user with a means to directly interact with the ferromagnetic fluid and the formation of shape changing buttons, while inheriting the aesthetic and kinetic qualities that Kodama's pioneering works offer.

Other ferrofluid interfaces include SnOil, which is a controllable display that allows the user to interact with ferromagnetic fluids by tilting the display [6]. The display uses a grid of electromagnets to actuate ferromagnetic fluid in selected areas, and a built-in tilt sensor is used to activate the electromagnets. Actuation of the ferrofluid is binary, off or on, depending on the respective electromagnet state. Linetic expand upon SnOil by offering a novel means of interaction with ferrofluid by directly inte-
racting with the material and provides an analog-like and linearly variable state of the actuated fluid, beyond simply a binary on/off state.

MudTub by Tom Gerhardt [7] is an intriguing, quasi-OUI that exploits the material characteristics of mud as an input material, but fails to follow the contemporary definition of OUIs as outlined by Holman and Vertegaal [8], in the sense that the interface medium and output display are not entirely the same. MudTub uses projection to augment the output and provide input. While offering a morphable computer interaction material in the form of mud is novel, our implementation offers more fluid-like properties, a higher degree of controllability, adheres to the tenants of OUIs as outlined by Holman and Vertegaal, and is not nearly as messy.

MudPad [11] is a haptic multi-touch input device that is malleable and can be controlled for localized haptic feedback. It acts similar to Linetic but the materials and methodology of actuation is different. The interaction material is also contained within a bladder and users do not directly interact with the material. Linetic offers the user direct interaction with the display and input material, making it a more compelling example of an OUI.

Programmable Blobs [24] is a compelling example of a user interface that uses malleable, programmable matter. It differs from Linetic in that it still uses a quasi-solid material compared to the much more fluid attributes of ferrofluid in the Linetic system. Latency during transformation of the Programmable Blobs is also much slower when compared to the speed of Linetic.

All the examples discussed above provide an invaluable and important basis for the research conducted for Linetic. Linetic attempts to expand the field of TUIs beyond solid and static-shaped object manipulation, bridging the gap between physical user actuation and animated representation into a unified input/output device.

## 3 System Description

A system diagram is shown in Figure 2 and the first version of the Linetic system is shown in Figure 1. By moving fingers over the ferrofluid buttons, users can manipulate shapes in the liquid in order to interact with the system. The system identifies a magnetic flux change when a finger moves perpendicular and vertically in relation to the surface. The flux change is detected by the Hall Effect sensors embedded beneath the surface of the ferrofluid, which sense the magnetic field produced by the magnets worn on the fingers. In turn, an electromagnet also contained beneath the surface of the ferrofluid is activated and produces a field in which the ferrofluid reacts to, producing a ferrofluid button. The distance between the finger and the ferrofluid button is mapped to the pressure and intensity of the click. Haptic feedback is felt by the user through the natural vibrations of the magnet worn on the fingertips. In the following section we will describe both the hardware (sensing and actuation) and software design of the system in detail. In essence, the Linetic system consists of a set of controller circuits to control the flow of the current, an array of Hall Effect sensors to detect user interaction, an array of electromagnets to actuate the ferrofluid, a ferrofluid pool,
a MAX/MSP sub-system for sound generation and a software application programming interface (API).

<figure><img src="image_3.png" /><figcaption>Fig.2. A System diagram of the Linetic system.</figcaption></figure>

**3.1 Hardware Design**

In the following section, we will outline the various hardware sub-systems of the Li-
netic system. Hardware sub-systems include the sensing method, the actuation acces-
sory, the electromagnetic array and circuit.

<figure><img src="image_4.png" /><figcaption>Fig.3. Interacting with version two of the Linetic system using the wearable accessory.</figcaption></figure>

**Sensing.** The hardware for sensing is performed by magnetic Hall Effect sensors manufactured by Hamlin Electronics. Hall Effect sensors measure the density of a magnetic field. The sensor produces an analog output when it is under the influence of any magnetic field. The sensor output is connected to the analog-to-digital conver-
ter of the micro controller used. The sensor is placed directly on top of each electro-magnet while the user's finger carries a powerful neodymium magnet with like poles of each magnet facing one another. When the electromagnet is turned on, the sensor output becomes fully saturated. If a neodymium magnet of the same pole is brought close to the surface, the sensor output drops and this is detected as the presence of the user's finger. Voltage drop of the sensor is dependent on the strength of the external magnetic field. Therefore, a sensor voltage drop is much higher when the distance between the finger and the ferrofluid display is reduced. This measurement is used as an input to control the height of the spikes, which are a product of the system.

**Actuation Accessory.** The actuation of the system, as seen in Figure 3, takes place when the user moves their fingers across the ferrofluid surface while wearing a magnetic finger accessory. To design this accessory, we looked at various accessories musicians wear on their fingers, such as guitar picks, to give us ideas on how to best design an accessory that acted as a natural extension of the finger. Achieving this kind of interaction was important for us so we decided to design a ring that allows the users to wear and play with the system. We also developed a wand that can be waved over the surface in order to provide an alternative method of interaction with the system but in this manuscript we will focus on the wearable configuration.

<figure><img src="image_5.png" /><figcaption>**Fig.4.** Design iterations for the wearable component. The final design is still being developed.</figcaption></figure>

We designed a ring that could be worn on the index finger of each hand. Figure 4 shows some of the design iterations. The rings contain a magnet that allows users to actuate the ferrofluid through natural gestures and finger movements. The ring has a cylindrical base made of plastic with a metal, coned shaped tip containing a magnet. When the users play with the system, they can sense a subtle haptic feedback through magnetic repulsion. This enables users to judge the distance from the liquid surface without touching it. This is important since when the user gets too close to the fluid, the liquid will become attracted to the metal cone. To avoid this, the haptic feedback enables the user to use the fluid interface without wetting or staining the fingers. Furthermore, electromagnet polarity is configured in such a way as to repel the magnet
on the user's finger, once again subtly directing the user to not place her hands too close to the liquid.

**Actuation.** The screen consists of ferrofluid placed in a shallow acrylic container. This container is placed on top of an array of electromagnets. These electromagnets, when turned on, actuate the fluid, causing it to form bumps and/or spiky buttons depending on the strength of the magnetic field produced. The system is powered using an array of ATX power supplies. These power supplies are then fed to a current limiting circuit. The output current from the current limiter circuits is distributed to the magnet driver board consisting of full-bridge drivers. By feeding the microcontroller's pulse-width modulation (PWM) signals to the magnet drivers, the electromagnetic field produced by the electromagnet array is controlled. This electromagnetic field creates spikes in various shapes and sizes on the surface of the ferrofluid display.

## 3.2 Software Design

In the next section, we will outline the various software sub-systems of the Linetic system. Software sub-systems include the Linetic OUI Framework, sound generation and the circuit firmware. Software sub-systems include the Linetic OUI Framework, sound generation and the circuit firmware.

**Linetic OUI Framework.** One of the objectives of the research is to develop a software framework for the project. It consists of a system background service and a Java API extension. The framework hides the complexity of the overall hardware and low-level software code from the developer and provides a convenient way to plug different sub-systems into the Linetic system. Therefore the Linetic system could be configured as a tangible input device, a tangible display, or perform both operations to the external system simultaneously. Various kinds of external systems could communicate with the Linetic system through the Linetic OUI framework API.

**Sound Generation.** In this application we attached Linetic to an instrumental music generation program developed in the MAX/MSP environment. The Max/MSP patch that is used detects incoming signals from the framework and activates appropriate midi piano keys to create sound.

**Circuit Firmware.** The circuit firmware is written in C and programmed into the microcontroller within the circuit. It communicates with the PC through USB interface and receives information regarding ferrofluid actuation patterns. According to the given information, it dynamically actuates the ferrofluid and creates dynamic patterns in the ferrofluid display by changing the current of the electromagnet grid underneath the ferrofluid container.
# 4 Experiments & Results

The previous work of authors includes a series of experiments to measure spike height versus current, distance of two adjacent spikes, transient state of the system and the static linearity of the system [1]. These experiments demonstrated the linearity of the control mechanisms, which are used in the Linetic system. Since magnetic systems are highly nonlinear, these experiments represent a significant step in controlling ferrofluids. The following experiments explore the control mechanisms of the system.

## 4.1 Experiment 1: Hall Effect Sensor Reading Versus Vertical Distance

<figure><img src="image_6.png" /><figcaption>Fig.5. Sensor output versus vertical distance.</figcaption></figure>

The point of this experiment is to understand the Hall Effect sensor readings in relation to the vertical distance of the actuator (in this case, the finger-worn actuation ring) from the surface. This experiment has been conducted using a Hall Effect sensor and an electromagnet, which generates an average flux density on the surface from 450 to 1950 Gauss for the range of 6V to 24V with 1.9A to 7.5A of electrical current. In the experiment we kept the power of the electromagnet at a constant voltage of 10V and a driven current of 2.44A, with the sensor on the vertical axis on top of the electromagnet. The sensor reading is measured versus the distance to the electromagnet. The value of the sensor output voltage taken is the mean value in one second.

This plot shows (Fig.5) that the sensor is most sensitive with respect to the vertical distance from 0cm to 3cm. When the distance is greater than 3cm, the change in output is much lower. At larger distances, for example the values of 6cm and 7cm, the difference in voltage is only 0.011 volts. However such a small voltage difference is not detected by the microcontroller used for this iteration of the system. Therefore,
the sensor cannot detect motion from 6cm to 7cm, but it can detect longer distance motion, e.g. moving from 6cm to 8cm.

## 4.2 Experiment 2: Hall Effect Sensor Reading Versus Horizontal Distance

This experiment is similar to this first experiment, but looks at horizontal distance as opposed to vertical distance. Once more keeping the power of the electromagnet constant, the sensor is placed on the vertical axis of the electromagnet at 2cm, since at this distance the sensor is most sensitive, registering the largest change in values with respect to the distance moved. An oscilloscope was used to measure the voltage output of the sensor.

<figure><img src="image_7.png" /><figcaption>Fig.6. Hall Effect sensor output versus horizontal displacement.</figcaption></figure>

Experiment 1 shows that the resolution of the system cannot distinguish any smaller changes. Within 0.02 volts, the magnetic field at 3.5cm and beyond is too small to cause a change in the microprocessor. This experiment shows that the magnetic field from the horizontal area of the magnet is too small to be detected at the optimal vertical distance. This is advantageous for the system since the tracking system gives an almost perfect horizontal reading without external noise from the neighboring magnets.

## 4.3 Experiment 3: Characterization of Magnetic Hall Effect Sensor Readings Under the Influence of Multiple Magnetic Fields

In this experiment the readings of the Hall Effect sensor are measured to determine the influence of the magnetic fields generated by the electromagnets and neodymium magnets. These readings will provide the appropriate parameters for the microcontroller firmware.
The electromagnet is connected to a power supply, supplying 10V and up to 2A of current. Its output magnitude is controlled by a Pulse-Width Modulated (PWM) input to the driver circuit. It is positioned such that the field of the North Pole is directed vertically upwards and the South Pole vertically downwards. The sensor is supplied the rated of 5V and is positioned such that it is in level with the top of the electromagnet and directly next to it. The reading of the steady-state output voltage of the sensor is recorded using the oscilloscope, while varying the height and direction of the neodymium magnet and the PWM input to the electromagnet. First the default sensor value is taken. Without the neodymium magnet or electromagnet influence, the reading is 2.50V. Next, with the neodymium magnet pole at South Pole (facing down), the PWM values and distances are measured.

<figure><img src="image_8.png" /><figcaption>Fig.7. Sensor reading values obtained for different distances versus PWM.</figcaption></figure>

Here (Fig.7) the strength of the electromagnet's field serves to decrease the reading of the sensor, whereas the position of the neodymium magnet field serves to increase the reading of the sensor. This results in a case where the value of the sensor is unable to detect the presence of the neodymium magnet due to the electromagnet's field. From the data we gathered, this occurs in the case when the distance of the neodymium magnet is 7.0cm. If the electromagnet is off, the reading is 2.53V, but if the electromagnet is turned on, the reading falls below the 2.50V neutral value. To circumvent this problem, we could use like poles instead of unlike poles, i.e. the North side of the neodymium magnet facing the North side of the electromagnet. This causes both magnets to boost each other's readings, which can be compensated for by changing the sensitivity in the software. This approach has the peripheral advantages of preventing the two magnets from attracting each other and preventing the neodymium magnet from picking up the ferrofluid contained on the surface of the system. It also provides a means for haptic feedback while using the system.
## 5 User Interaction Methodology

In regards to the user interaction methodology, we attempted to simulate some interactions that reflect both the aesthetics and playfulness of interacting with fluid. Although the dramatic effects of water as a tactile surface has been previously explored [19], no precedent in regards to the interaction with ferrofluid, which has distinct fluid characteristics, has been recorded. Due to this, an adaptation of previous methods as well as trial and error was employed.

<figure><img src="image_9.png" /><figcaption>Fig.8 Version Three of the Linetic system without the splash cover.</figcaption></figure>

Due to the messy nature of ferrofluid as well as the nature of sensing inherent to the system, we decided to employ a finger accessory that allowed users to interact with the fluid without actually touching it. This reduced the methods of interaction to that if simple mid-air gestures, some of which are employed in [1]. These gestures included waving and tapping. Like-polarity between the surface and fingertip magnets allowed a force resistance that made tapping a particularly compelling interaction.

Later on, we decided to remove the haptic subsystem in order to concentrate on the visual and audio effects of the system. This led us to the third iteration of the Linetic system.

## 6 Aesthetic Interactions with Linetic

Although efficiency and speed are two important characteristics for most technical devices, research suggests that these attributes are not sufficient to fulfill all of a user's needs [4] [16]. Aside from the technical characteristics of the device, a user needs a series of other characteristics that enrich the user experience of an artifact, device or technology. People search for these characteristics that allow exploration and a holistic experience of an artifact instead of just basic features. Through Linetic, we hope to tap into these deeper desires and human emotions in order to be able to produce a
more enriching, aesthetic experience of the system. The aesthetic experience refers to
the pragmatic aesthetics that Dewey [3] and Shustermann [21] describe, as well as the
guidelines that Petersen and others have developed for Aesthetic Interactions [18]
from the pragmatist aesthetics viewpoint.

Fels [5] states that for an individual to have an aesthetic experience there needs to be an intimacy that is built between the person and the object. Fels believes that when people are able to manipulate an object skillfully or intimately, this produces an aesthetic experience. He describes four types of relationships that one is able to have with an object and these are as follows: the person communicates with the object; the person embodies the object; object communicates with person; object embodies the person. Through Linetic we are able to achieve the first three of these experiences: person communicates with object; object communicates with person; the person embodies the object.

First, the user is able to communicate with the system in a direct and natural way through touch. Interacting with the liquid, the user can start a dialogue in which the system responds to the user. The still, dark pool of ferrofluid, with its reflective surface like a black mirror, has the ability to draw people to it in the same way artist Richard Wilson [26] was able to capture the imagination of his audience with his installation 20:50 - a room filled with thick sump oil. According to Khaslavsky et al. [12], the first step to seduce your audience is to entice them in order to grab their attention. It is the very liquid quality of ferrofluid that makes it a material that probes curiosity and encourages interaction. This curiosity naturally leads to one wanting to touch the liquid and explore it.

The Linetic system possesses the playful characteristics needed for learning by playing, which is a learning mechanism for most people. The system lends itself to playful exploration and thus a relationship with the system can begin. Unlike a solid tangible interface, liquid has the ability to take the shape of its container and allows for the free maneuvering of itself. As people move their fingers above the liquid they begin to learn quickly how to sculpt and manipulate the liquid, creating three dimensional, liquid sculptures combined with sound that respond to the user's tactile exploration. This seemingly magical quality of being able to sculpt ferrofluid has been the main spark of inspiration that motivates us to exploit this interaction.

Aesthetic interaction suggests that the use of the system gives it meaning and creates an aesthetic experience [19], the appropriation of Linetic by the user and the freedom for improvisation makes this a platform for creative expression. Here the user is able to intelligently construct music through the simple and ordinary actions of playing with liquid, making the natural into a magical experience, thus capturing their imagination.

The third way in which a person can relate to an object is when the object communicates to the person; there is no interaction and it is much like how we enjoy a painting. Even at this level Linetic has the ability to engage the audience. The liquid, which seems to have a life of its own, can mesmerize even non-interacting audiences and this quality was evident in the works of Sachiko Kodama. With Linetic, the audience can reach out to the liquid and with the movement of their fingers, can engage with this liquid as a piano player would with piano keys. This allows the user to en-
gage in a more intimate way with the interface as Fels defines intimacy, and allows the user to use this interface as an extension of them.

One of the questions we wanted to address with this project was how we can keep audiences engaged in the experience, and help them take away a meaningful experience. Music or art that has stood the test of time are examples of works where the creator is able to captivate the audience even after decades from when the work was produced [17]. Many interactive systems today have not yet been able to capture users in the same way. People easily adapt to new technologies and once they understand how to use it, it can or frequently loses its novelty. Liquid is a material that people from different disciplines continue to study to understand its powerful effect on our body, mind and beyond. Liquid as an interface for computing opens the possibility of a whole new way of how we understand and engage with the computing world. The exact ways in which we might use liquids in future interactive systems is still to be explored and Linetic is a beginning of this exploration.

## 7 Characterizing Linetic as Disposed to Analogness or Digitalness

Using the method offered by Koh et al. [15], we attempted to situate Linetic in the Analogness-Digitalness Continuum (ADC), by comparing the system with the characteristics for interactive systems offered in the research. Table X shows a breakdown of how Linetic scored in regards to its Analogness & Digitalness.

<table><thead><tr><td>Characteristics of Analogness & Digitalness</td><td>Affirmative</td></tr></thead><tbody><tr><td>The Interface is Intuitive</td><td></td></tr><tr><td>Interaction is Continuous</td><td>✓</td></tr><tr><td>Content and Media are Singular in Embodiment</td><td>✓</td></tr><tr><td>Content and Media are Dichotomistic in Embodiment</td><td></td></tr><tr><td>Interaction is Discrete</td><td>✓</td></tr><tr><td>The Interface is Mediated</td><td></td></tr><tr><td>Hybrid Candidate?</td><td>✓</td></tr></tbody></table>
<caption>**Table 1.** A table showing how Linetic scored in terms of its disposition to Analogness or Digitalness.</caption>

In Analysis of how Linetic scored, we found that our current version of the interaction required some training before being able to use the system and thus did not meet the requirements of an initiative interface.

However Linetic did meet the requirements for a continuous interface where media and content are a singular embodiment, as the morphable ferrofluid used in our interface is both a visual component of information feedback to the user (in addition to sound and formally, haptics), as well as the interface where the user interacts with the system.

Curiously depending on the configuration of Linetic, the interaction characteristics could become discreet. This is mainly due to the possibility to use Linetic as an interface device to an external system.

In conclusion, we find Linetic to lean towards the Analogness side of the continuum, with hybrid-like qualities depending on the configuration of the system.

## Future Work

Future work for this project involves efforts in several directions: exploration of new materials, new array configurations, and improvements in the methodology of gestural interactions and most importantly, the development of relevant applications in order to explore the full potential of human-computer interaction using liquids as a medium.

The search for new materials for different effects and increased controllability involves experimentation with materials such as electrorheological (ER) and magnetorheological (MR) fluids. These materials will provide another avenue for OUIs and the Linetic project.

Refinement of the electromagnet array is also important so as to create a more fine-grained and higher-resolution display. The use of other magnet arrangements will also enable us to control electromagnetic fields better, so as to create more complex and intricate shapes, which will enable us to explore more real-world applications.

An effort to create a concise library of gestures that are more intuitive for use with the system is also desirable in order to disseminate a methodology in which other researchers can base new research on. Once a concise gestural library is created the project will focus on developing and executing more user studies in order to fine-tune the usability of the system. Finally an integrated development environment (IDE) for direct programming of Linetic is desired so that the technology can be shared internationally, which can foster application development in the academic and commercial markets.

## 8 Conclusion

In this manuscript we describe new iterations of the Linetic system that uses the Hall Effect as a novel form of interaction and actuation. By using Hall Effect sensing in
combination with a new accessory worn on the fingertips, users avoid messiness and staining, which was a previous downfall of the original system.

We also outline in this paper, three new experiments that enabled us to develop the new iterations of the Linetic system. These include findings regarding the relationship between perpendicular distances of the Hall Effect sensor and the magnetic field generated by the electromagnetic array embedded in the surface of the system, the characterization of the magnetic Hall Effect sensor readings under the influence of multiple magnetic fields, and the relationship of distance from the sensor versus PWM.

A preliminary user study was also performed in order to understand the usability of the system, as well as to inform the authors on the possible challenges of evaluating such a system for use as an interface and display device. We also discuss the addition of haptic feedback facilitated by the vibration of the magnets placed on the fingertips, providing an additional modality of feedback, along with the previously developed modalities of sound and visual output. This new actuation accessory provides a means for users to interact with the Linetic system without the need to actually touch the ferrofluid, and still provides instantaneous tactile feedback.

Finally we also discuss the aesthetic implications in regards to the user's experience when using the Linetic system. The aesthetic implications of using liquid as a means for human-computer interaction are an important topic in regards to our research. We felt a need to outline these implications in order to accurately prepare for future liquid-based interactive systems.

## Acknowledgement

This research is supported by the Singapore National Research Foundation under its International Research Center Keio-NUS CUTE Center @ Singapore Funding Initiative and administered by the IDM Program Office.

## References

1. Bailly, G., Vo, D.B., Lecolinet, E., Guiard, Y.: Gesture-aware remote controls: guidelines and interaction technique, in: Proceedings of the 13th international conference on multi-modal interfaces, ICMI '11, ACM, New York, NY, USA, pp. 263–270 (2011)

2. Coelho, M., Maes, P.: Sprout I / O: A Texturally Rich Interface, ACM, pp. 221–222 (2008)

3. Dewey, J.: Art as experience, volume 12, Perigee, (1934)

4. A. Dunne, Hertzian Tales. Electronic Products, Aesthetic Experience, and Critical Design, volume 2nd, RCA Computer Related Design Re- search, (1999)

5. Fels, S.: Intimacy and Embodiment: Implications for art and technology, Computer Engineering 13–16 (2000)

6. Frey, M.: Snoil. a physical display based on ferrofluid. http://www.freymartin.de/en/projects/snoil/, 2006. [Online; accessed 18-August-2011].

7. Gerhardt, T.: Mud tub, http://tomgerhardt.com/mudtub/, [Online; accessed 18-August-2011]. (2009)
8. Holman, D., Vertegaal, R.: Organic user interfaces: designing computers in any way, shape, or form, Communications of the ACM 51 (2008) 4855.

9. Ishii, H.: Tangible bits: designing the seamless interface between people, bits, and atoms, ACM, p. 3.

10. Ishii, H.: The tangible user interface and its evolution, Communications of the ACM 51 (2008) 32.

11. Jansen, Y.: Mudpad: Fluid haptics for multitouch surfaces, Human Factors 4351–4356. (2010)

12. Khaslavsky, Shedroff, N.: Understanding the seductive experience, Communications of the ACM 42 pp. 45–49 (1999)

13. Kodama, S.: Dynamic ferrofluid sculpture, Communications of the ACM 51 (2008) 79.

14. Koh, J.T.K.V., Karunanayaka, K., Sepulveda J. et al.: Liquid interface: a malleable, tran-sient, direct-touch interface. In Proceedings of the 7th International Conference on Ad-vances in Computer Entertainment Technology - ACE '10 (2010)

15. Koh J.T.K.V., Peiris, R.L., Zhu, K., Polydorou, D., Nakatsu, R.: Uncovering analogness and digitalness in interactive media. In *Proceedings of the 30th ACM international conference on Design of communication* (SIGDOC '12). ACM, New York, NY, USA, 233-242.C. (2012)

16. Korsmeyer, Picard, R.W.: affective computing, Minds and Machines 9 (1999).

17. Locher, P., Overbeeke, K., Wensveen, S.: A framework for aesthetic experience, Advances 9–12 (2009)

18. Petersen, M. G., Iversen, O. S., Krogh, P. G., Ludvigsen, M.: Aesthetic interaction, ACM Press, (2004)

19. Pier, M. D., Goldberg, I. R.: Using water as interface media in vr applications, in: Proceedings of the 2005 Latin American conference on Human-computer interaction, CLIHC '05, ACM, New York, NY, USA, pp. 162–169 (2005)

20. Schwesig, C.: What makes an interface feel organic?, Communications of the ACM 51 (2008) 67.

21. Shusterman, R.: Pragmatist aesthetics; living beauty, rethinking art, volume 50, Blackwell, (1992)

22. Vertegaal, R., Poupyrev, I.: Introduction (special section on organic user interfaces), Communications of the ACM 51 pp. 26–30 (2008)

23. Wakita, A., Nakano, A., Kobayashi, N.: Programmable blobs, ACM Press, (2011)

24. Weiser, M.: The computer for the 21st century, Humancomputer interaction: toward the year 2000 pp. 933–940 (1995)