# A Framing and Analysis of Applicative Tangible Interfaces

Guillaume Rivière

► To cite this version:

Guillaume Rivière. A Framing and Analysis of Applicative Tangible Interfaces. 2025. hal-05454950

HAL Id: hal-05454950

https://hal.science/hal-05454950v1

Preprint submitted on 12 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# A Framing and Analysis of Applicative Tangible Interfaces

Guillaume Rivière

g.riviere@estia.fr

Univ. Bordeaux, ESTIA Institute of Technology

Bidart, France

## Abstract

The investigation of tangible user interfaces commenced approximately thirty years ago. Questions on its commercial potential become more pressing as the field becomes mature. To take the field one step further—as the emergence of components contributed to the commercial development of graphical user interfaces—this article suggests that applicative tangible user interfaces could also be split into components. These components are composed of the aggregation, combination, or coupling of physical items and fulfil four roles that are described through a new interaction model. This article successfully distributed among these four components’ roles all of the 159 physical items from a representative collection of 35 applications. Further examination of these applicative tangible interfaces coincides with four research phases in the field and identifies three main paths for future research to fully realize the potential of tangible user interfaces.

## CCS Concepts

* Human-centered computing → HCI theory, concepts and models.

## Keywords

tangible user interfaces, tangible components, applications, interaction model, taxonomy, roles, bodiness, classification, analysis

## ACM Reference Format:

Guillaume Rivière. 2025. A Framing and Analysis of Applicative Tangible Interfaces. *arXiv.org, Cornell University, Ithaca, NY, USA.* 16 pages.

## 1 Introduction

Since the conceptualization of tangible user interfaces (TUIs) in 1997 [51], researchers have produced several hundred specimens over the past few decades. For instance, the Tangible Media Group at the MIT Media Lab alone generated an impressive 218 specimens<fnref n="1" /> between 1997 and 2023 (i.e., an average of eight specimens yearly across twenty-six years). This landscape of tangible interfaces comprises prototypes showcasing new technologies (e.g., The Actuated Workbench [66]), and others exhibiting concrete applications of the concept (e.g., Computational Optimization [67]). While the number of prototypes grew yearly, taxonomies, paradigms, and frameworks emerged to facilitate a better understanding of tangible user interfaces [42, 64, 85: Ch. 5, 98: Ch. 3]. In 2010 and, more

recently, in 2022, an extensive review [85] and a thorough book [98] provided comprehensive overviews of the field. With increasing maturity, there is growing scrutiny of the market viability and realism of tangible interfaces [38, 39]. When examining the history of graphical user interfaces (GUIs), the process of standardization and industrialization began with the imagination of the WIMP (Windows, Icons, Menus, Pointer) components<fnref n="2" /> [39, 88, 89] and their simplification [39] in the early 1980s, which took advantage of the mouse invention over a decade earlier [23]. Stimulated by this history, this article suggests studying the components found in the applicative tangible interfaces that have been proposed in the literature by raising the following questions. Could we split applicative tangible user interfaces into components? Are all these components required to develop applicative tangible user interfaces? Are there blind spots in tangible components that require further investigation before tangible interfaces could become ready for industrial development?

Previous work have framed the landscape of tangible user interfaces to better understand the degrees of tangibility and revealed trends in three same applicative domains to reach higher degrees of tangibility [24]. This work frames applicative tangible user interfaces at the level of their tangible components, which are revealed from the aggregation of physical items into roles that are articulated through a new interaction model. Describing a collection of 35 applicative tangible user interfaces (that deal with different application domains, or deal with same application domains differently) enables retrieving and confirming four development phases of the field, as well as scrutinizing underexplored opportunities of components for richer interaction and more complete applications, and providing insights for further exploring the components required for applicative tangible interfaces. Furthermore, these analyses are refined through four classes of bodiness that are defined for applicative tangible user interfaces according to the physical items involved in the implementation of their tangible data components.

First, this article provides an overview of the roles imagined in earlier models, notations, and toolkits, along with a summary of taxonomies and classifications regarding tangible interfaces. Subsequently, it establishes a novel taxonomy of physical items by relying on a new interaction model and constructs a classification of applicative tangible interfaces based on this taxonomy. It then showcases the taxonomy and the classification by detailing a collection of 35 applications from the literature and examining interfaces' designs and the field's evolution. Finally, it concludes by addressing some future directions for the field and outlining some limitations of the work.

<figure><img src="image_2.png" /></figure>

<footnote n="1">¹See the project page of the Tangible Media Group: https://tangible.media.mit.edu/projects/ (last accessed on August 31, 2023).</footnote>

<footnote n="2">²The origins of WIMP components date back to as early as 1968 or possibly even 1945 [35] (as cited in [39]).</footnote>
## 2 Related Work

Models of interaction and interfaces abstract the entities involved in interaction by assigning them roles. Then, interaction and interfaces can be described, understood, and compared across paradigms. Efforts to better understand tangible interfaces also call for taxonomies and classifications of these user interfaces. This section reviews (1) the roles described in some previous models, notations, and toolkits and (2) some taxonomies and classifications of tangible interfaces.

### 2.1 Roles in Models, Notations, and Toolkits

Instrumental interaction [5] is an interaction model in which “instruments” are used to edit “domain objects” (i.e., data role) attributes. These instruments consist of a physical part (an input device) and a logical part (a graphical representation of the instrument) that translate user actions into commands that edit domain objects. Instruments provide the user with reactions from the instrument on its actions and feedback from edited object responses. In time-multiplexed conditions, activating the instrument entails associating the physical and logical parts [5]. For example, the instrument may be activated by directly pointing at it or clicking a button in a tool palette [5]. Instrumental interaction permits the description and comparison of WIMP interaction and interaction with graspable user interfaces [26] within a unified interaction model [5].

In the mixed interaction model [15], instruments are refined as “mixed tools” (i.e., tools role), and domain objects become “task objects” (i.e., data role), both of which are categorized as “mixed objects.” *Mixed objects* possess physical and logical properties that are interconnected through “linking modalities” [15]. *Input linking modalities* sense physical proprieties using input devices and convert them into digital properties through *input languages* [15]. Conversely, *output linking modalities* utilize *output languages* and devices to translate digital properties into physical properties [15]. Hence, where instrumental interaction deals only with domain objects shown graphically (on monitors or tabletops), mixed interaction extends to “bodied task objects” (i.e., bodied data) that can provide input and output at the same time.

ASUR++ [20, 21] is an extension of the ASUR notation [22] that characterizes the physical and digital entities of mobile mixed systems (i.e., Adapter, System, User, and Real object). The notation is also suitable for tangible user interfaces [20]. The notation distinguishes between two digital roles for physical objects by defining terms for tools (“Rtool”) that aid in a task (i.e., tools role) and targeted objects (“Robject”) that are altered by a task (i.e., data role).

The ROSS API [104]—Responsive Objects, Surfaces, and Spaces— outlines four nested levels for tangible interface elements, including the description of physical objects (“RObject”), as well as active spaces (“RSpace”) and surfaces (“RSurface”) capable of sensing interaction. Thus, a key advantage of this low-level framework is its ability to go beyond physical objects’ description. However, even if input controls such as buttons, switches, and sliders are described (“RControl”), the API does not include descriptions of the roles of physical objects such as data and tools. Nonetheless, the toolkit’s description of surfaces—which indicate where physical

objects are placed and where graphics are displayed and touched, such as tabletop surfaces and mobile phone screens—suggests that limiting to the two roles of data and tools may be insufficient for fully describing interaction with tangible interfaces.

Data and tools also feature in tangible interfaces’ paradigms [40, 86, 92, 96], in addition to operations and constraints. In the Token-Based Access to Digital Information [40], “tokens” are bodied items of specific form to manipulate digital information (i.e., data role) within applications, where as “containers” are bodied items of generic form to move any information between devices and applications, “faucets” are intangible items that convey digital information, and “tools” are bodied items bound to any computational functions without distinction between data editing (e.g., physical handles to scale graphical shapes [26]) and application features (e.g., applying filters to data rendering with the passive lens [51]). In the Token+Constraint approach [92, 96], “tokens” are bodied items that represent digital information (i.e., data role) and “constraints” are visually or mechanically delimited regions (i.e., constraints role) that are bound to computational operations (e.g., show, save). These operations are triggered when a token is sensed for presence, translation, or rotation within the constrained region. In the TAC Paradigm [86], “tokens” refer to the physical representations of digital information (i.e., data role) or computational functions (i.e., tools role), and “constraints” refer to the physical items that structure the user’s manipulation of the tokens (i.e., constraints role).

Thereby, the different roles that are found in tangible user interfaces can be summarized as data, tools, operations, and constraints. However, interaction models, notations, toolkits, and paradigms describe physical items only in the range of data, tools, and constraints (see Table 1). Operations with the application (e.g., show, browse, color mode) or the operating system (e.g., load, save, authentication) are melted with tools that edit data (e.g., in [40]) or bound to some trigger conditions between physical items such as tokens and constraints [86, 92, 96]. Nevertheless, the tangible components assigned to the operations’ role have later been coined and defined as “core tangibles” and “domain tangibles” [97], which are tangible components that implement operations shared among multiple applications (e.g., load, save, authentication) or some application domains (e.g., setting color in drawing applications), regardless of the application’s interaction style [97].

To expound upon these models, notations, and paradigms, this article relies on the four roles of data, tools, operations, and constraints to introduce a role-based interaction model, and then a taxonomy and a classification based on these roles.

### 2.2 Taxonomies and Classifications

The literature has utilized taxonomies and classifications to chart the conceptual space of tangible user interfaces. The seminal work on graspable user interfaces presented a conceptual space with thirteen axes [26]. Later, Ishii categorized tangible user interfaces into eight genres [44]. Various frameworks—approximately two dozen in total—have organized the conceptual space along different axes [64, 85: Ch. 5, 98: Ch. 3]. Those frameworks organized tangible interfaces by abstracting, designing, or building their experiences, domains, physicality, interactions, or technologies [64].
<table><caption>Table 1: Interface components' roles among some models, notations, and paradigms in tangible and mixed reality scopes.</caption><thead><tr><th colspan="3">Previous Work</th><th colspan="4">Components' Roles</th></tr><tr><th>Name</th><th>Year</th><th>References</th><th>Data</th><th>Tools</th><th>Operations</th><th>Constraints</th></tr></thead><tbody><tr><td>Instrumental Interaction</td><td>1997</td><td>[4, 5]</td><td>Domain objects</td><td>Instruments</td><td>-</td><td>-</td></tr><tr><td>Token-Based Access</td><td>1999</td><td>[40]</td><td>Containers, Tokens, Faucets</td><td>Tools</td><td>-</td><td>-</td></tr><tr><td>Token+Constraint</td><td>2002</td><td>[92, 96]</td><td>Tokens</td><td>-</td><td>-</td><td>Constraints</td></tr><tr><td>ASUR++</td><td>2002</td><td>[20, 21]</td><td>R<sub>object</sub></td><td>R<sub>tool</sub></td><td>-</td><td>-</td></tr><tr><td>TAC Paradigm</td><td>2004</td><td>[86]</td><td>Tokens</td><td>Tokens</td><td>-</td><td>Constraints</td></tr><tr><td>Mixed Interaction</td><td>2006</td><td>[15]</td><td>Task objects</td><td>Mixed tools</td><td>-</td><td>-</td></tr></tbody></table>

For instance, Fishkin's taxonomy of tangibility [24] has organized tangible interfaces as a conceptual space of twenty categories along two axes: embodiment (i.e., the physical relationship between the input focus and the output focus of a tangible interface) and metaphor (i.e., the analogy between user actions or artifact meaning compared with real-world actions or objects, respectively). The analysis of nine applicative tangible interfaces—that were selected because dealing with the same three task domains among over 60 tangible user interfaces—revealed a trend toward seeking a higher degree of tangibility [24]. The next section develops a new taxonomy that categorizes the physical items that constitute tangible interface components.

## 3 Proposing a Taxonomy and a Classification

This section provides a taxonomy of the physical items that are utilized in tangible user interface's components, based on their bodiness, by relying on four tangible-component roles found in applicative tangible interfaces, and introduces a new interaction model based on the relationships between these roles, before it splits applicative tangible interfaces into four bodiness classes.

The concept of a “tangible user interface” is considered here broadly to include any user interface that strives for a “seamless coupling between bits and atoms” using “atoms” as representations and controls of digital information and functions [51]. Such tangible and related physical user interfaces range from “passive real-world props” [37] to “graspable user interfaces” [26], “manipulative user interfaces” [34], and “embodied user interfaces” [25]. Without a difference, those approaches were simply advancements within a developing field [24], which combined in 1997 under the cohesive vision of “tangible bits” [51]. Furthermore, in this article, the term “bodied” replaces the term “physical” when referring to some graspable elements; then the term “physical” encompasses both graspable and non-graspable entities (such as graphics, video projection, and audio).

However, before introducing the two axes of the taxonomy, this section priorly requires a clarification of the three following concepts: entities, components, and items.

### 3.1 Entities, Components, and Items

Let us first define the set of the physical interactional entities that are involved in tangible and related physical user interfaces. “Physical Interactional Entities” are considered the highest level of abstraction of anything that is generated, animated, or sensed by computing resources and that is perceivable, editable, or

manipulable by users. Such “Interactional Entities” are composable, couplable, and hybridable. “Physical” comprises matter (e.g., objects, sides, and substances), sound (e.g., audio, percussion, and noise), and any graphical item (e.g., shapes, texts, and images). For example, the sides of a cube can be considered as entities, as well as the whole cube, depending on what is sensed, mapped, or represented. Some other entities are added to this initial set, which are not generated, nor animated, nor sensed by computing resources, but which support, hold, or guide entities from the initial set during interaction (e.g., surfaces, slots, and trails).

The term “component” refers to the entities that fulfil roles in user interaction, such as windows, icons, menus, and pointers components in the WIMP paradigm for graphical user interfaces. In addition, components are made of “items.” As an illustration, in the WIMP paradigm, window components basically consist of items such as an outer border, header bar, title, close button, and content pane. Likewise, desktops' icon components consist of two items: a pictogram and a caption.

In applicative tangible user interfaces, an interactive role in the user dialogue with computing resources can be insured by a single interactional entity itself, or by several interactional entities that are aggregated, combined, or coupled to fulfil this whole role. Thereby, a “tangible component” is such an interactional entity that fulfil an interactive role, or a subset of the interactional entities that fulfil an interactive role together. The “physical items” of a tangible user interface are any one-piece interactional entity whose bodiness can be defined and that are involved into interactive roles. Those physical items comprise elements that are sensed or controlled by computing resources, as well as some that are not (i.e., even if they do not directly represent digital information or functions, e.g., when insuring some mechanical or structural functions). As a consequence, a tangible component can aggregate, combine, or couple several physical items (of same or different bodinesses) to fulfil its role (e.g., a bodied cursor and a bodiless value are coupled to constitute the color map amplitude tool of Teegi [33]), as well as a tangible component can be composed of a single physical item. However, the number of physical items associated to the hierarchy of a component can also depend on the description's level of granularity (such as with the nested levels of the ROSS API [104]). This hierarchy is summarized in Figure 1.

The next subsection details the first axis of the taxonomy, which delineates four roles for tangible interface components.
<figure><img src="image_3.png" /><figcaption>**Figure 1:** The hierarchy of the elements that compose applicative tangible interfaces.</figcaption></figure>

## 3.2 Four Components' Roles

The first axis concerns components' roles in which physical items are involved. This axis comprises four values drawn from prior framings of tangible interfaces, mixed reality, and instruments: data, tools, operations, and constraints. The relationships between these four roles articulate through the DTOC role-based interaction model that is depicted in Figure 2. These roles are taken as a sufficient basis to describe the components involved in applicative tangible user interfaces specifically.

The role of *data* refers to what users sense, observe, or modify with the aid of computing resources; data justify the use or presence of such computing resources. Users perceive data through data items that serve as data presentations. In some cases, users can edit data directly through data presentations (i.e., direct manipulation [87]), and computer-generated feedback informs users about data modification. Theoretically, edited data are stored in long-term computer memory after the interaction ends. The role of data is demonstrated in various models, notations, toolkits, and paradigms, such as the instrumental interaction model (i.e., “domain object”) [5], the mixed interaction model (i.e., “task object”) [15], the ASUR notation (i.e., “R<sub>object</sub>”) [20, 21], and tangible interface paradigms (i.e., “tokens”) [40, 86, 92, 96].

Tangible interfaces have explored the use of physical items for several data types. Spatial data, such as terrain relief, can be manipulated directly by hand through the use of clay [73]. Abstract data, such as the intensity of car traffic or stock exchange rates, can be conveyed by the rotation speed of Pinwheels [103]. Likewise, the locations of mobile phone antennas can be represented by symbolic pucks on a two-dimensional city map and be moved by computing resources for optimization concerns [67]. Sometimes, data items are not editable on their own (i.e., incapable of input), but bodied tools then allow for manipulating and modifying data.

Tools enable users to edit data. The role of tools also appears in various models, such as the instrumental interaction model [5] (i.e., “instrument”), the mixed interaction model [15] (i.e., “mixed tool”), and the ASUR notation [20, 21] (i.e., “R<sub>tool</sub>”). For instance, in GraspDraw [26], digital anchors in the form of bricks allow users

<figure><img src="image_4.png" /><figcaption>**Figure 2:** The DTOC interaction model of tangible components' roles in applicative tangible interfaces.</figcaption></figure>

to resize, rotate, and translate geometric shapes (i.e., data editing) directly through graphical items displayed on a desk surface. In Tangible Geospace [51], a device for scaling and rotating allows for browsing a campus map presentation (i.e., data presentation editing). In Teegi [33], a cursor enables the indirect editing of the visualization color scheme's amplitude (i.e., data representation editing).

Operations provide access to the application and operating system layers. A first role of *operations* is to trigger events and actions within applications and operating systems. “Operating system operations” cover a broad set of commands, ranging from loading or saving data [96, 97] to starting or closing applications [97] and authenticating or disconnecting users [102]. “Application operations” allow users to alter or customize how the interactive systems behave, including the activation or parameterization of tools, the change of tool modes, the clutch or declutch of tools and data items, and the validation of actions. For instance, TUISTER [11] enables the operation of hierarchical structures navigation across several applications. GraspDraw [26] features an inkwell to set the current draw color mode. In Head Prop for Neurosurgical Visualization [37], a thumb button and a foot pedal are used for clutching operations of the plate tool and doll's head data items, respectively. However, operations are also about exhibiting the outcomes and calculations provided by the applications and operating systems. For instance, the application of the Urban Planning workbench [6, 45, 99] provides features to measure wind
magnitude and the distance between buildings, which require displaying the calculated values. Another example consists of the four measurements<fnref n="3" /> that are provided by the application of the IP Network Design Workbench [59], which are monitored on the vertical graph screen.

*Constraints* structure interaction by guiding motion, restricting position, and organizing data, tools, or operations items. The concept initially arises as “reference frames” within the early tangible user interfaces framework [94] and later manifests as “constraints” within Token + Constraints systems [92, 96] and the Token And Constraints paradigm [86]. Constraints can be physical or digital.

“Physical constraints” are imposed on bodied items before these items are sensed by computing resources. For instance, a rack orders a series of operations to process in a specific order in the Slot Machine [72]. These constraints exerted on bodied data thus indirectly fulfil a digital role by structuring mapped digital information. Physical constraints can be visual, kinesthetic, mechanical, or dynamic:

(a) “Visual constraints” are bodiless items. They are imposed on bodied items by the users’ cognition. For example, when users place pucks on a grid drawn on a whiteboard [53] or when users target displayed circles with assisted pucks [55].

(b) “Kinesthetic constraints” are bodied items. They are exerted on bodied items by users’ cognition and sense of kinesthesia. For instance, when users place plastic cards into the rows’ slots of the Slot Machine [72].

(c) “Mechanical constraints” are static bodied items, but they are imposed on the bodied items that are enacted or actuated by computing resources. The path that guides released balls in the Marble Answering Machine (see in [30, 51, 98: Ch. 1]), or a flexible curve that prevents actuated antenna representations from entering a specific area in PICO [67] are examples of such mechanical constraints.

(d) “Dynamic constraints” are mechanical constraints that are exerted on some bodied items by dynamic bodied items (e.g., moving or morphing constraints during the application’s runtime). For example, a morphing surface that enables lifting, translating, or tilting some bodied items [30].

Like in constraint-based approaches [86, 92, 96], physical constraints can eventually be associated with a set of functions that are activated when spatial concordance of some bodied items within the confining regions and that are called when sensing some physical properties of these bodied items (e.g., position, translation, or rotation). According to the bodied component that is present, these functions can relate to data, tools, or operations.

“Digital constraints” (also known as software constraints [67]) are exerted by computing resources on bodied items to maintain consistency with their digital state. For instance, if users disturb the equal distance between three pucks, computing resources will restore digital constraints by actuating the pucks’ positions [67]. As such, digital constraints are not physical items but the computer-generated forces exerted on bodied items.

## 3.3 The Bodiness of Physical Items

The second axis relates to the *bodiness* of physical items. The three possible values comprise: static body, dynamic body, and bodiless.

*Bodiless* items cannot be grasped. Such physical items encompass graphics, drawings, and sounds. Whereas the supports of graphical items are touchable (e.g., surfaces), the graphical items themselves cannot be grasped (i.e., held and manipulated) or sensed kinesthetically. Consequently, visual attention must be consistently maintained, preventing opportunities for eye-free operations [36]. These visual items may be computer-generated graphics or handcrafted drawings on a surface. For example, the grid drawn on the Senseboard [53] whiteboard serves as a bodiless constraint. The two-dimensional brain slice displayed on the distant monitor of the Head Prop for Neurosurgical Visualization [37] is a bodiless datum item. The amplitude level displayed on the table surface of Teegi [33] is a bodiless tool item. Likewise, audio information, which cannot be grasped is a bodiless item. Therefore, bodiless items comprise “non-graspable” [94] and “intangible” [92, 96] representations.

Physical items with static or dynamic bodies can be grasped, as they occupy a volume in three-dimensional space—the difference between the two lies in the animation of dynamic-bodied items by computing resources. The body of *static* items is inert: users can move it (e.g., the bricks of GraspDraw [26]), rotate it (e.g., the doll’s head and the cutting plate in the Head Prop for Neurosurgical Visualization [37]), and deform it (e.g., the malleable surface of Illuminated Clay [73] and the flexible curves of PICO [67]). However, computing resources do not affect it. Conversely, computing resources actuate the body of *dynamic* items. For instance, computing resources move the antenna representations of PICO [67], roll the cylinders of inTouch [9], spin the blades of Pinwheels [103], and alter the shape of Relief [60, 61].

Consequently, this taxonomy establishes a conceptual space consisting of twelve categories that account for components’ roles and their items’ bodiness. The remainder of this section builds upon this taxonomy to classify tangible interfaces into four classes.

## 3.4 Four Role-Centered Bodiness Classes

This section suggests classifying applicative tangible interfaces based on the bodiness of data items:

* *Class I specimens* exclusively rely on bodied data, using either static or dynamic bodied data items without bodiless data items.

* *Class II specimens* rely on a combination of bodiless data items with either static or dynamic bodied data items.

* *Class III specimens* are based on the combination of bodied tools with bodiless data (i.e., bodied data are absent).

* *Class IV specimens* only provide bodied operations (i.e., no data) that are not restricted to a single application and can control the same shared operations across applications.

Table 2 presents the occurrence patterns of item kinds in each class. In addition, we can rediscover some of the major research inspirations in tangible interfaces in these four classes:

* Pure (or “full” [24]) tangible user interfaces (i.e., the most tangible ones, in regard to the “tangible bits” vision [51] and the tangibility spectrum [24]) belong to Class I;

* Passive real-world props [37] belong to Class II;

<footnote n="3">³The four measurements of the IP Network Design Workbench [59] are “client response time,” “protocol response time,” “link usage rate,” and “monthly running cost.”</footnote>
<table><thead><tr><th colspan="2">Physical Item Kind</th><th colspan="4">Bodiness Class</th></tr><tr><th>Role</th><th>Bodiness</th><th>Class I</th><th>Class II</th><th>Class III</th><th>Class IV</th></tr></thead><tbody><tr><td rowspan="2">Data</td><td>Bodied</td><td>+</td><td>+</td><td>0</td><td>0</td></tr><tr><td>Bodiless</td><td>0</td><td>+</td><td>+</td><td>0</td></tr><tr><td rowspan="2">Tools</td><td>Bodied</td><td>*</td><td>*</td><td>+</td><td>0</td></tr><tr><td>Bodiless</td><td>*</td><td>*</td><td>*</td><td>0</td></tr><tr><td rowspan="2">Operations</td><td>Bodied</td><td>*</td><td>*</td><td>*</td><td>+</td></tr><tr><td>Bodiless</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td rowspan="2">Constraints</td><td>Bodied</td><td>*</td><td>*</td><td>*</td><td>*</td></tr><tr><td>Bodiless</td><td>*</td><td>*</td><td>*</td><td>*</td></tr></tbody></table>

Note. "+" = some items (one or more). "-" = none or some items.

<caption>Table 2: Definition of four bodiness classes according to the physical item involved in the four components' roles.</caption>

* Graspable user interfaces [26] belong to Class III;

* Core tangibles [97] belong to Class IV.

The following section analyzes the literature by using the taxonomy to describe the items from a sample of applications, which are then classified into the four classes.

# 4 Analyzing Applicative Tangible Interfaces

This section selects a collection of applicative tangible interfaces from the literature that deal with various application domains, and describes the items used in this collection through the taxonomy and sorts the applications into the four classes. Then, it scrutinizes the evolution of the field and the design of applicative tangible interfaces by examining the representative sample of specimens.

## 4.1 Selection Process of the Collection

The current collection is intended to be merely a partial summary of hundreds of tangible interface specimens that researchers produced in the past few decades. Specimen selection is less strict than in systematic literature reviews, which hinders the performance of precise statistics. The goal of the selection is to encompass a variety of emblematic tangible interface specimens and offer a representative overview alongside some insights into the field. Restricting the selection to a specific date range may have led to the exclusion of certain tangible interface kinds, as research interests fluctuate over the years. Thus, the entire research period was taken into account, from premises to recent work. Six relevant sources were browsed:

* The project page<fnref n="4" /> of the MIT Media Lab's Tangible Media Group (TMG), which is recognized as a prominent and prolific group in the exploration of tangible user interfaces. The final collection includes specimens of this provenance in 43% of the cases.

* The sessions with “tangible,” “embodied,” “physical,” or related terms in their title from eighteen ACM CHI conference proceedings spanning from 2005 to 2022 venues<fnref n="5" />;

<table><caption>Table 3: Composition of the collection of 35 specimens from six sources, showing overlaps between sources.</caption><thead><tr><th colspan="2">Source</th><th colspan="7">Specimens' Inclusion</th></tr><tr><th>Name</th><th>Period</th><th>Once</th><th>Twice</th><th>Thrice</th><th>Quarce</th><th>Quince</th><th>Sence</th><th>Total</th></tr></thead><tbody><tr><td>TMG</td><td>1990-2023</td><td>1</td><td>2</td><td>3</td><td>4</td><td>5</td><td>0</td><td>15</td></tr><tr><td>CHI</td><td>2005-2022</td><td>2</td><td>2</td><td>4</td><td>6</td><td>5</td><td>0</td><td>19</td></tr><tr><td>TEI</td><td>2007-2023</td><td>1</td><td>2</td><td>3</td><td>1</td><td>0</td><td>0</td><td>7</td></tr><tr><td>[98]</td><td>1976-2022</td><td>1</td><td>5</td><td>0</td><td>5</td><td>5</td><td>0</td><td>16</td></tr><tr><td>[85]</td><td>1976-2010</td><td>0</td><td>5</td><td>7</td><td>5</td><td>5</td><td>0</td><td>22</td></tr><tr><td>[82]</td><td>1991-2009</td><td>3</td><td>0</td><td>4</td><td>7</td><td>5</td><td>0</td><td>19</td></tr><tr><td>All</td><td>1976-2023</td><td>8</td><td>8</td><td>7</td><td>7</td><td>5</td><td>0</td><td>35</td></tr><tr><td></td><td></td><td>23%</td><td>23%</td><td>20%</td><td>20%</td><td>14%</td><td>0%</td><td></td></tr></tbody></table>

* The seventeen ACM TEI conference proceedings, from 2007 to 2023 venues;

* Three states of the art, from one book devoted to the field in 2022 [98], one major review of the field in 2010 [85], and one literature study that focused on applicative tangible interfaces in 2009 [82].

The sources were screened to retain work dealing with applications. Work solely demonstrating a new technology, interaction technique, or tangible interface kind was excluded because describing them with the DTOC role-based interaction model makes no sense. Entries were filtered based on their name, thumbnail image, or publication title. Therefore, entries that provided a sneak peek of an application in their name, thumbnail image, or publication title were given precedence. Publications and demonstration videos of eligible work were downloaded and filtered to ensure adequate descriptions of applications, user interfaces, and interactions. Only applications with new application domains were included in the collection, unless they had different implementations from other ones. Table 3 provides a summary of the six sources' inclusion in the collection.

## 4.2 Applications' Labeling and Classification

Inventorying and labeling of the physical items were carried out by reading their descriptions in publications and watching any available supplementary demonstration videos. Tables 4, 5, and 6 report the resulting labeling of 159 physical items from the 35 applications of the collection. The domains and bodiness classes are listed in Table 7: the collection comprises 4 applications of Class I, 21 of Class II, 5 of Class III, and 5 of Class IV.

## 4.3 Taxonomy Coverage

All the four roles are valuable in describing the 159 physical items required by the 35 applications. In addition, the four roles proved to be adequate in capturing all of these physical items. As illustrated in Figure 3, the role of data was utilized the most (71 items), followed by tools (34 items), operations (30 items), and constraints (24 items).

However, only ten of the twelve taxonomy categories are utilized: dynamic-bodied items are absent from the operations and constraints roles of the collection; dynamic-bodied items are found only for data and tools roles (see Figure 4). Moreover, data are mainly implemented through bodiless items, whereas tools,

<footnote n="4">⁴The project page of the Tangible Media Group displays a chronological list of specimen names and corresponding thumbnail images: https://tangible.media.mit.edu/projects/ (last accessed on August 31, 2023).</footnote>

<footnote n="5">⁵As of August 31, 2023, the titles of ACM CHI conference sessions are available on the ACM Digital Library website, https://dl.acm.org/, from the year 2005 onward. No dedicated session was found for the year 2023.</footnote>
<table><thead><tr><th colspan="4">Tangible Interface</th><th colspan="10">Physical Items</th></tr><tr><th rowspan="2">#</th><th rowspan="2">Name</th><th rowspan="2">Year</th><th rowspan="2">References</th><th rowspan="2">Name or Description</th><th colspan="4">Role</th><th colspan="3">Bodiness</th></tr><tr><th>D</th><th>T</th><th>O</th><th>C</th><th>S</th><th>D</th><th>L</th></tr></thead><tbody><tr><td>1.</td><td>Slot Machine</td><td>1976</td><td>[72]</td><td>Rows (procedure)<br>Buttons (on each row)<br>Plastic cards (commands)<br>Display triangle (turtle)<br>Circular robot (turtle)</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>-</td></tr><tr><td>2.</td><td>CAAD 3D Modelling System</td><td>1979</td><td>[1, 2]</td><td>Building Blocks<br>Design geometry (perspective view)<br>Evaluation measures (Isoplots)</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>3.</td><td>Self-Builder Model (Segal Model)</td><td>1980</td><td>[31, 32] cited from [90]</td><td>Panels<br>Board<br>Wireframe rendering<br>Feedback tool (house area, cost)</td><td>X</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>4.</td><td>Marble Answering Machine</td><td>1992</td><td>See in [30, 51, 98: Ch. 1]</td><td>Machine's path (queue)<br>Machine's indentation (play slot)<br>Marble (message)<br>Message</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>5.</td><td>Head Prop</td><td>1994</td><td>[37]</td><td>Doll's head (brain)<br>Plate (slicing)<br>Plate thumb button (clutch)<br>Foot pedal (clutch)<br>Brain 2D slice</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>6.</td><td>GraspDraw</td><td>1995</td><td>[26]</td><td>Bricks<br>2D shapes<br>Inkwell<br>Functions' tray (select, delete, shapes)<br>ActiveDesk surface</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>7.</td><td>Tangible Geospace</td><td>1997</td><td>[51]</td><td>Phicons<br>Campus Map<br>Overlay view (passive lens)<br>3D view (active lens)<br>Scaling and Rotating Device<br>Desk's surface</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>8.</td><td>BUILD-IT</td><td>1997</td><td>[28, 79, 80]</td><td>Bricks<br>Plan view<br>Objects (robots, tables...)<br>Virtual cameras<br>EyeCatchers<br>Table surface<br>3D view</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>9.</td><td>Pinwheels</td><td>1998</td><td>[103]</td><td>Pinwheels</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>10.</td><td>mediaBlocks</td><td>1998</td><td>[93, 95]</td><td>mediaBlock (storage)<br>Slots</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>11.</td><td>musicBottles</td><td>1999</td><td>[46, 48]</td><td>Music<br>Bottle (music)<br>Cork<br>Triangular table<br>Central "stage" area</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>12.</td><td>Urp (Urban Planning Workbench)</td><td>1999</td><td>[6, 45, 99]</td><td>Architectural Model<br>Road-object (strips)<br>Material-transformation-object (wand)<br>Video-camera-object<br>Clock-object<br>Wind-generating tool<br>Anemometer-object (arrow)<br>Wind magnitude (number)<br>Distance-measuring-object<br>Distance (number)<br>Shadows/Reflections<br>Airflow grid<br>Workbench</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>13.</td><td>Senseboard</td><td>2001</td><td>[53]</td><td>Grid<br>Rectangular pucks<br>View detail puck<br>Arrow puck<br>Values<br>Board</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>X</td></tr><tr><td>14.</td><td>Illuminating Clay</td><td>2002</td><td>[49, 73]</td><td>Clay Model<br>Rotative platform</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td><td>X</td></tr></tbody></table>

Note. "-" = empty. Role: "D" = Data, "T" = Tool, "O" = Operation, "C" = Constraint, Bodiness: "S" = Static, "D" = Dynamic, "L" = Bodiless.

<caption>Table 4: The physical items of thirteen applicative tangible interfaces (ordered by ascending years, from 1976 to 2001).</caption>
<table><thead><tr><th colspan="3">Tangible Interface</th><th colspan="7">Physical Items</th></tr><tr><th>#</th><th>Name</th><th>Year</th><th>References</th><th>Name or Description</th><th colspan="4">Role</th><th colspan="3">Bodiness</th></tr><tr><th></th><th></th><th></th><th></th><th></th><th>D</th><th>T</th><th>O</th><th>C</th><th>S</th><th>D</th><th>L</th></tr></thead><tbody><tr><td></td><td></td><td></td><td></td><td>Crosshairs</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Cross Sections</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Analysis Function Thumbnails</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>3-D perspective view</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td>15.</td><td>Audiopad</td><td>2002</td><td>[68, 69]</td><td>Sounds</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Pucks (audio tracks)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Star-shape puck (sound selector)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Hierarchical menu</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>SenseTable surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td>16.</td><td>reacTable</td><td>2003</td><td>[56]</td><td>Music</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Point (audio output)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Square puck (audio source)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Rounded square puck (filter)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Round puck (controller)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Decagon puck (control filter)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Pentagon puck (audio mixer)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Lines (audio flow)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Table surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td>17.</td><td>IP Network Design Workbench</td><td>2003</td><td>[59]</td><td>Pucks</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Nodes</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Links</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Nodes menu</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Parameter puck (with button)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Link bandwidth</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Router service priority</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Number of client users</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Server performance</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Table surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Measurement graphs</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td></tr><tr><td>18.</td><td>Query Shapes</td><td>2004</td><td>[43]</td><td>Cubes (ActiveCubes)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Voxel data representation</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>3D shape models</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td>19.</td><td>TUISTER</td><td>2004</td><td>[11]</td><td>TUISTER</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>20.</td><td>I/O Brush</td><td>2004</td><td>[83, 84]</td><td>Brush</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>World (palette)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Canvas</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td>21.</td><td>PICO</td><td>2005</td><td>[67]</td><td>Pucks (antenas)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Map (city)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Flexible curve</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Rubber band</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Collar</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Teflon / Sandpaper</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Table surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td>22.</td><td>Augmented Chemistry</td><td>2007</td><td>[27]</td><td>Booklet</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Gripper</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Cube</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Augmented 3D view</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td>23.</td><td>Nimio</td><td>2007</td><td>[10]</td><td>Pyramid Nimios (color-changing)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Cube Nimios (color-changing)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Dome Nimios (color-changing)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Cylinder Nimios (color-changing)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>24.</td><td>ArcheoTUI</td><td>2007</td><td>[81]</td><td>Props</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>3D fragments</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Foot pedals (clutch)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>25.</td><td>GeoTUI</td><td>2008</td><td>[16]</td><td>Map (cube top view)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Cutting line</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Two-puck prop</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Ruler prop</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Button box</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td></td><td></td><td></td><td></td><td>Tabletop</td><td>-</td><td>-</td><td>-</td><td>X</td><td>X</td><td>-</td><td>-</td></tr><tr><td>26.</td><td>Slurp</td><td>2008</td><td>[107]</td><td>Eyedropper</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>27.</td><td>Relief</td><td>2010</td><td>[60, 61]</td><td>2.5D shape display (terrain)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Topographical map</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td>28.</td><td>Bidirectional mixiTUI</td><td>2011</td><td>[70, 71]</td><td>Sounds (Music)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr><tr><td></td><td></td><td></td><td></td><td>Tracks (vibrating lines)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td></tr></tbody></table>

Note. "-" = empty. Role: "D" = Data, "T" = Tool, "O" = Operation, "C" = Constraint. Bodiness: "S" = Static, "D" = Dynamic, "L" = Bodiless.

<caption>Table 5: The physical items of fourteen applicative tangible interfaces (ordered by ascending years, from 2002 to 2010).</caption>
<table><thead><tr><th colspan="4">Tangible Interface</th><th colspan="5">Physical Items</th></tr><tr><th rowspan="2">#</th><th rowspan="2">Name</th><th rowspan="2">Year</th><th rowspan="2">References</th><th rowspan="2">Name or Description</th><th colspan="4">Role</th><th colspan="3">Bodiness</th></tr><tr><th>D</th><th>T</th><th>O</th><th>C</th><th>S</th><th>D</th><th>L</th></tr></thead><tbody><tr><td rowspan="10">29.</td><td rowspan="10">Teegi</td><td rowspan="10">2014</td><td rowspan="10">[33]</td><td>Active loop tokens (Precomposed samples)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Loop tokens graphical feedback</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Active effect tokens</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Effect tokens graphical feedback</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Control tokens (Session, Stop)</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Colored tabs (sessions selectors)</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Teegi character (user's activity)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>Brain model (user's activity)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>Filter area</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Mini-Teegis (filters)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td rowspan="5">30.</td><td rowspan="5">SoundFORMS</td><td rowspan="5">2016</td><td rowspan="5">[13]</td><td>EEG Raw Signals</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Color map amplitude</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Color map cursor</td><td>-</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Table surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Trigger pins</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td rowspan="2">31.</td><td rowspan="2">CairnFORM</td><td rowspan="2">2019</td><td rowspan="2">[18]</td><td>Soundwave pins</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Sounds</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>32.</td><td>reSpire</td><td>2019</td><td>[12]</td><td></td><td>Ring chart</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>33.</td><td>Embodied Axes</td><td>2020</td><td>[14]</td><td></td><td>Shape-changing fabric</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td rowspan="5">34.</td><td rowspan="5">CoDa</td><td rowspan="5">2020</td><td rowspan="5">[101]</td><td>Orthogonal arms (data axes)</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>Tokens (data points)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>Interactive surface</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td><td>-</td></tr><tr><td>Sidebar buttons (filter and analytic functions)</td><td>-</td><td>X</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Data points</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td rowspan="2">35.</td><td rowspan="2">SABLIER</td><td rowspan="2">2022</td><td rowspan="2">[63]</td><td>Analytical functions (lines)</td><td>X</td><td>-</td><td>-</td><td>-</td><td>-</td><td>X</td><td>-</td></tr><tr><td>Hourglass</td><td>-</td><td>-</td><td>X</td><td>-</td><td>X</td><td>-</td></tr></tbody></table>

Note. "-" = empty. Role: "D" = Data, "T" = Tool, "O" = Operation, "C" = Constraint. Bodiness: "S" = Static, "D" = Dynamic, "L" = Bodiless.

<figure><img src="image_5.png" /><figcaption>Figure 3: Distribution through the four roles of the 159 physical items from the collection.</figcaption></figure>

operations, and constraints are mainly implemented through static-bodied items.

## 4.4 Class Distribution Over Time

The application's distribution over time through the four classes provides insights into the research phases in the field of tangible interfaces. To this end, Figure 5 depicts and clusters the distribution of the 35 applications from the collection by class and time. A highly productive phase spans over fifteen years from the mid-1990s to the late 2000s, with 66% of the applications ranging from 1992 to 2008 (23 out of 35). All four classes have applications during this phase. This abundance must align with a research endeavor to prove the utility of the novel technologies introduced by the emergence of the tangible interface concept. At present, the exploration of

<figure><img src="image_6.png" /><figcaption>Figure 4: Bodiness distribution across the four roles in the 159 physical items from the collection of applications.</figcaption></figure>

Class III applications appear to remain limited to this phase, with the majority of specimens concentrated on examining tangible tabletops (four out of five applications involved horizontal surfaces). Subsequently, the frequency of applications decreased during a
<table><caption>Table 7: Application domains and bodiness classes of the 35 applications.</caption><thead><tr><th>#</th><th>Application</th><th>Domain</th><th>Class</th></tr></thead><tbody><tr><td>1.</td><td>Slot Machine</td><td>Programming</td><td>II</td></tr><tr><td>2.</td><td>CAAD 3D Modelling System</td><td>Architecture</td><td>II</td></tr><tr><td>3.</td><td>Self-Builder Model (Segal Model)</td><td>Architecture</td><td>II</td></tr><tr><td>4.</td><td>Marble Answering Machine</td><td>Telecommunication</td><td>II</td></tr><tr><td>5.</td><td>Head Prop</td><td>Medicine (Surgery)</td><td>II</td></tr><tr><td>6.</td><td>GraspDraw</td><td>Drawing</td><td>III</td></tr><tr><td>7.</td><td>Tangible Geospace</td><td>Geography</td><td>II</td></tr><tr><td>8.</td><td>BUILD-IT</td><td>Architecture</td><td>III</td></tr><tr><td>9.</td><td>Pinwheels</td><td>Communication</td><td>I</td></tr><tr><td>10.</td><td>mediaBlocks</td><td>Ubiquitous Computing</td><td>IV</td></tr><tr><td>11.</td><td>musicBottles</td><td>Music Listening</td><td>II</td></tr><tr><td>12.</td><td>Urp (Urban Planning Workbench)</td><td>Town planning</td><td>II</td></tr><tr><td>13.</td><td>Senseboard</td><td>Scheduling</td><td>II</td></tr><tr><td>14.</td><td>Illuminating Clay</td><td>Town planning</td><td>II</td></tr><tr><td>15.</td><td>Audiopad</td><td>Live music synthesizing</td><td>II</td></tr><tr><td>16.</td><td>reactTable</td><td>Live music synthesizing</td><td>II</td></tr><tr><td>17.</td><td>IP Network Design Workbench</td><td>Information Technology</td><td>III</td></tr><tr><td>18.</td><td>Query Shapes</td><td>3D Shapes Databases</td><td>II</td></tr><tr><td>19.</td><td>TUISTER</td><td>Ubiquitous Computing</td><td>IV</td></tr><tr><td>20.</td><td>I/O Brush</td><td>Painting</td><td>III</td></tr><tr><td>21.</td><td>PICO</td><td>Town planning</td><td>II</td></tr><tr><td>22.</td><td>Augmented Chemistry</td><td>Organic Chemistry</td><td>II</td></tr><tr><td>23.</td><td>Nimio</td><td>Group Awareness</td><td>I</td></tr><tr><td>24.</td><td>ArcheoTUI</td><td>Archaeology</td><td>II</td></tr><tr><td>25.</td><td>GeoTUI</td><td>Geosciences</td><td>III</td></tr><tr><td>26.</td><td>Slurp</td><td>Ubiquitous Computing</td><td>IV</td></tr><tr><td>27.</td><td>Relief</td><td>Topography</td><td>II</td></tr><tr><td>28.</td><td>Bidirectional mixiTUI</td><td>Live music rearranging</td><td>II</td></tr><tr><td>29.</td><td>Teegi</td><td>Brain activity</td><td>II</td></tr><tr><td>30.</td><td>SoundFORMS</td><td>Live music synthesizing</td><td>II</td></tr><tr><td>31.</td><td>CairnFORM</td><td>Sustainability</td><td>I</td></tr><tr><td>32.</td><td>reSpire</td><td>Health (Mindfulness)</td><td>I</td></tr><tr><td>33.</td><td>Embodied Axes</td><td>Augmented Reality Spaces</td><td>IV</td></tr><tr><td>34.</td><td>CoDa</td><td>Mathematics</td><td>II</td></tr><tr><td>35.</td><td>SABLIER</td><td>Archaeology</td><td>IV</td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 5: Distribution of the 35 specimens, by class and time.</figcaption></figure>

<table><thead><tr><th colspan="2">Physical Item Kind</th><th colspan="4">Bodiness Class</th></tr><tr><th>Role</th><th>Bodiness</th><th>Class I</th><th>Class II</th><th>Class III</th><th>Class IV</th></tr></thead><tbody><tr><td rowspan="3">Data</td><td>Static</td><td>I</td><td>||||||||||||</td><td>Ø</td><td>Ø</td></tr><tr><td>Dynamic</td><td>III</td><td>||||</td><td>Ø</td><td>Ø</td></tr><tr><td>Bodiless</td><td>Ø</td><td>||||||||||||||||</td><td>||||</td><td>Ø</td></tr><tr><td rowspan="3">Tools</td><td>Static</td><td>o</td><td>||||||||</td><td>||||</td><td>Ø</td></tr><tr><td>Dynamic</td><td>o</td><td>I</td><td>o</td><td>Ø</td></tr><tr><td>Bodiless</td><td>o</td><td>III</td><td>II</td><td>Ø</td></tr><tr><td rowspan="3">Operations</td><td>Static</td><td>o</td><td>||||||||</td><td>||||</td><td>||||</td></tr><tr><td>Dynamic</td><td>o</td><td>o</td><td>o</td><td>o</td></tr><tr><td>Bodiless</td><td>o</td><td>III</td><td>II</td><td>o</td></tr><tr><td rowspan="3">Constraints</td><td>Static</td><td>o</td><td>||||||||||||</td><td>||||</td><td>I</td></tr><tr><td>Dynamic</td><td>o</td><td>o</td><td>o</td><td>o</td></tr><tr><td>Bodiless</td><td>o</td><td>II</td><td>I</td><td>o</td></tr></tbody></table>

Note. "I" = one application using at least one of item of this kind. "o" = no application using an item of this kind. "Ø" = this item kind is unavailable in this class.

<caption>Table 8: Use of item kinds by class in the 35 applications.</caption>

less productive phase in the 2010s due to increased emphasis on developing novel technologies for actuated and shape-changing tangible interfaces [47, 74, 77]. Accordingly, the resulting more scarce applications, concentrated on classes I and II, appear to be more focused on finding applications for actuation and shape change. From 2010, applications resorting to dynamic-bodied items account for five out of seven, whereas they account for only three out of twenty-three from 1992 to 2008. Finally, in the early 2020s, exploration of Class IV applications resumed after a decade-long hiatus, with a last phase comprising two specimens implementing operations in augmented reality and virtual reality scopes (i.e., Embodied Axes [14] (#33) and SABLIER [63] (#35), respectively).

## 4.5 Item Distribution Over Classes

Analyzing item's bodiness give some insights into the design trends among the four classes. To this end, Table 8 presents the utilization of item kinds in the 35 applications from the collection, categorized by bodiness class.

Tools are absent from the four Class I applications, which focus on dynamic-bodied data that are noneditable, either by implementing a tool or by itself. Nimio items [10] (#23) can sense input on their own, but are static-bodied data. In contrast, Class II applications appear to frequently implement tools; bodiless data items must amplify the presence of tools, as also observed in Class III applications. Likewise, operations are absent from Class I applications but are frequent in Class II and Class III applications. The increased presence of tools and operations within Class II and Class III applications indicates richer interaction and more complete applications compared to Class I. Most tools, operations, and constraints consist of static-bodied or bodiless items; tools made of dynamic-bodied items are utilized by only one of the 35 applications (i.e., bidirectional mixiTUI [70, 71] (#28)).

Thereby, the taxonomy and the classification afford the accurate framing and detailed analysis of 35 applications from the literature and their 159 items, and reveal some design opportunities that remain to explore. The next section offers perspectives on the field.
# 5 Discussion

This section provides insights into the evolution of the field and the future exploration of tangible interface design. Additionally, it addresses some limitations of the present analysis.

## 5.1 Evolution of the Research Field

The main effort in finding new applications for the tangible interface concept slowed down fifteen years ago. Since then, recent efforts have focused on taking advantage of dynamic-bodied items. Nevertheless, the scarcity of applications since 2010, compared to the two preceding decades, suggests that the community could now be more active in this pursuit. Potential challenges that hinder progress involve expertise, finances, and time:

(a) Developing tangible interface applications with dynamic-bodied items requires HCI students and researchers to integrate additional skills beyond computer programming, physical prototyping, and user-centered design. They must become more knowledgeable about sensing and actuating possibilities [8] and deepen their collaboration with engineers from mechanics and mechatronics.

(b) Physical computing is more expensive than graphical computing [39], especially when actuation capabilities are required. For instance, whereas a small 12×12 array of actuated pins already necessitates 144 motors, doubling lines' length to get a 24×24 array, like the one in SoundFORMS [13](#30), balloons to 576 motors, thus increasing cost by 4.

(c) The same, the development time of such dynamic interfaces increases as the intricacy grows very quickly, for example, according to the number of spare parts to produce, store, and assemble, and the number of cable of the wiring to cut, prepare, and mount. As well, writing the driver for new hardware adds to the applications' development time. All these steps render design and prototyping iterations even more challenging.

This way, the exploration of technical solutions to reduce the number of components and development time is still pending. Such solutions will not only contribute to decreasing the environmental impact but also improve economic viability [39, 40]. Only then, researchers could intensify the exploration of Class I and Class II applications and resume the exploration of Class III applications. To sustain this effort, three levers are necessary:

(1) Breaking the dependence on manufactured bodied items (e.g., liberated pixels [39], constructive inputs [62, 106], and molecular user interfaces [75, 76]);

(2) Reducing the technological cost and complexity of actuation and shape-change mechanisms [3, 54], for example, by investing the low tech aspiration [7, 65];

(3) Encouraging publications that illustrate design concepts of such applications—even if only through sketches, simulations, and interaction vocabularies [19, 41, 78]— can provide valuable guidance for further technological investigations.

Additionally, the increased financial and environmental costs associated with the development of bespoke hardware for tangible interfaces [38, 39] compared to graphical interfaces suggest to not

only focus on the embodiment of new data [39]. A promising path lies in the adjunction of meaningful bodied tools and operations to pixel-based interfaces other than tangible interfaces [38, 39]. The recent resurgence of Class IV exploration embraces this emerging trend by adding bodied operations to two augmented reality and virtual reality applications (i.e., Embodied Axes [14](#33) and SABLER [63](#35), respectively). A third recent example in 2019 is an old-fashioned hotel bell for front desks that allows guests to call a remote receptionist who is viewed on a monitor [38]. Such examples of Class IV applications still require further development, and should benefit from collaboration with researchers who focus on other interface scopes such as graphical interfaces, virtual reality, and augmented reality.

## 5.2 Perspectives for Design

Surprisingly, Class I applications appear to be infrequent, despite being the most aligned with the original “tangible bits” vision [51]. Additionally, these applications are predominantly composed of noneditable dynamic-bodied data. For instance, editable dynamic-bodied data, including shape-changing items—which require more extensive integration of user input capabilities [77]—are still untapped in Class I applications. To design richer interaction in such applications, future work must investigate self-editable dynamic-bodied data or combine dynamic-bodied data with bodied tools. In addition, constraints are primarily utilized in Class II and Class III applications. Investigating the use of constraints can also aid in the design of more sophisticated interactions in Class I and Class IV applications.

The implementation of dynamic-bodied tools, operations, and constraints should be further explored to potentially improve application in all of the four bodiness classes. Technology is ready, such as shape-changing buttons [91], expandable dials [58], and expandable pucks [57, 100], that need to benefit to Class II and Class III applications. For example, integrating active pucks [71] into mixiTUI [70] enhances interaction by providing haptic feedback for use ranges (i.e., to keep between minimum and maximum values), predefining computer-generated puck rotations (to play pre-composed effects), and grouping pucks for rotation imitation (to synchronize two audio tracks). Moreover, dynamic-bodied constraints still need to be explored. As an illustration, AeroRigUI [105] proposes a movable surface that can serve, when appropriate, as a separation or a projection screen. Another illustration is simulating the Marble Answering Machine with inFORM by dynamically shaping the constraints of the tangible user interface thanks to the array of pins [30]. This way, dynamic constraints enable switching between applications and tasks, which is also a promising direction for the commercial viability of tangible interfaces.

## 5.3 Limitations of the Analysis

A limitation of this work is the use of a taxonomy to describe the specimens without consultation with their designers. Describing the 35 applications was relatively straightforward, but at times, advanced questioning was required to comprehend their designers' thinking. For instance, the Pinwheels specimen [103](#9), which represents information flow ambiently, can be interpreted from two
viewpoints. At first glance, Pinwheels appear to consist of bodiless data items, where airflow represents the data flow. However, this idea has evolved from using airflow in the ambient ROOM [52], simply requiring air to enter through a hole, to using “spinning pinwheels” [17, 50, 103] as a visual representation. As a result, Pinwheels are now referred to as bodied data items.

Another example regards the inclusion of CairnFORM [18](#31) in Class I, but the exclusion of Relief [60, 61](#27). CairnFORM utilizes bodied rings to convey data through a combination of shape and color changes. Because color is determined by the physical properties of the rings, CairnFORM’s energy forecast was considered as a Class I application. Relief also combines shape and color changes of a bodied item to convey a topographic map. However, because the map display exceeds the boundaries of the bodied shape, topography encoding with Relief was considered a Class II application. Without this factor, Relief would have fallen into Class I. Therefore, knowing whether the overflow is a deliberate design choice, as seen in overview+detail or focus+context interfaces, holds implications for classification considerations. Nevertheless, despite being essential for generalization and standardization, these classification considerations remain secondary to the prevailing concerns of user experience and performance, which must primarily guide user interface design.

Therefore, advanced questioning can become challenging, as design thoughts are not always detailed in the publications. The community should benefit from future work that dissect prior specimens in a publicly accessible database (e.g., [29]), and from future publications that clearly describe items’ bodiness and the roles in which they are involved.

# 6 Conclusion

This article confirms four distinct phases in the emergence of tangible interface applications: spanning from the premises laid in the late 1970s and early 1980s to the productive exemplification of the concept in the 1990s and 2000s, the tight integration of actuation and shape-changing technologies in the 2010s, and the emerging integration in pixel-based interfaces in the early 2020s. A taxonomy outlining the physical items present in applicative tangible interface enhances understanding of the four phases and uncovers overlooked possibilities. The taxonomy consists of two axes: the roles of applications’ interactive components and the bodiness of the physical items involved in these components (considering that bodiless items—such as graphics and sound—are physical). The four roles of data, tools, operations, and constraints, which are articulated within a new interaction model for applicative tangible interfaces, accurately discriminated the 159 physical items from a collection of 35 applications from the literature. Building upon the taxonomy, four bodiness classes are identified according to how the data role is implemented.

Three potential paths for future tangible interface applications are promising. (a) The first path is to broaden the scope of Class I applications to editable dynamic-bodied data items through the implementation of self-editable data or the addition of bodied tools. (b) The second path is to gain leverage of dynamic-bodied tools, operations, and constraints that are currently underrepresented

in applicative tangible interfaces despite mature technology. This effort must directly benefit to applications from Classes I, II, and III. Three drivers support this effort: (1) breaking the dependence on manufactured bodied items, (2) reducing the technological cost and complexity of actuation and shape change mechanisms, and (3) encouraging publications that illustrate the design concepts of such applications, even if only through sketches, simulations, and interaction vocabularies. (c) Finally, the third path is to bring tangibility to pixel-based interfaces by implementing meaningful bodied operations from Class IV applications in scopes beyond tangible interfaces, including graphical interfaces, augmented reality, and virtual reality. Advancing along these three paths ought to contribute to the fulfillment of the Tangible Bits vision and get tangible interfaces a step closer to their commercial availability and viability.

# References

[1] Robert Aish. 1979. 3D input for CAAD systems. Computer-Aided Design 11, 2 (March 1979), 66-70. https://doi.org/10.1016/0010-4485(79)90098-8

[2] Robert Aish and Peter Noakes. 1984. Architecture without numbers – CAAD based on a 3D modelling system. Computer-Aided Design 16, 6 (Nov. 1984), 321-328. https://doi.org/10.1016/0010-4485(84)90116-7

[3] Jason Alexander, Anne Roudaut, Jürgen Steimle, Kasper Hornbæk, Miguel Bruns Alonso, Sean Follmer, and Timothy Merritt. 2018. Grand Challenges in Shape-Changing Interface Research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal, Quebec, Canada, April 21-26, 2018) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3173574.3173873

[4] Michel Beaudouin-Lafon. 1997. Interaction instrumentale : de la manipulation directe à la réalité augmentée. In Actes des Neuvumièmes Journées sur l'Ingénierie des Interfaces Homme-Machine (Poitiers, France, September 10-12, 1997) (IHM '97). Cépaduès-Editions, Toulouse, France, 97-104.

[5] Michel Beaudouin-Lafon. 2000. Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (The Hague, The Netherlands, April 1-6, 2000) (CHI '00). Association for Computing Machinery, New York, NY, USA, 446-453. https://doi.org/10.1145/332040.332473

[6] Eran Ben-Joseph, Hiroshi Ishii, John Underkoffler, Ben Piper, and Luke Yeung. 2001. Urban Simulation and the Luminous Planning Table: Bridging the Gap between the Digital and the Tangible. Journal of Planning Education and Research 21, 2 (Dec. 2001), 196-203. https://doi.org/10.1177/0739456X0102100207

[7] Philippe Bihoux. 2020. The Age of Low Tech: Towards a Technologically Sustainable Civilization. Bristol University Press, Bristol, UK.

[8] Laurens Boer, Harvey Bewley, Ben Cahill, and Steffen Hvíd Trier. 2021. Smörgåsbords for Physical Computing. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (Salzburg, Austria, February 14-17, 2021) (TEI '21). Association for Computing Machinery, New York, NY, USA, Article 101, 12 pages. https://doi.org/10.1145/3430524.3446077

[9] Scott Brave and Andrew Dahley. 1997. inTouch: A Medium for Haptic Interpersonal Communication. In CHI '97 Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA, March 22-27, 1997) (CHI EA '97). Association for Computing Machinery, New York, NY, USA, 363-364. https://doi.org/10.1145/1120212.1120435

[10] Johanna Brewer, Amanda Williams, and Paul Dourish. 2007. A Handle on What's Going on: Combining Tangible Interfaces and Ambient Displays for Collaborative Groups. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (Baton Rouge, Louisiana, USA, February 15-17, 2007) (TEI '07). Association for Computing Machinery, New York, NY, USA, 3-10. https://doi.org/10.1145/1226969.1226971

[11] Andreas Butz, Markus Groß, and Antonio Krüger. 2004. TUISTER: A Tangible UI for Hierarchical Structures. In Proceedings of the 9th International Conference on Intelligent User Interfaces (Funchal, Madeira, Portugal, January 13-16, 2004) (IUI '04). Association for Computing Machinery, New York, NY, USA, 223-225. https://doi.org/10.1145/964442.964486

[12] Kyung Yun Choi, Valentina Sumini, and Hiroshi Ishii. 2019. reSpire: Self-Awareness and Interpersonal Connectedness through Shape-Changing Fabric Display. In Proceedings of the 2019 on Creativity and Cognition (San Diego, California, USA, June 23-26, 2019) (C&C '19). Association for Computing Machinery, New York, NY, USA, 449-454. https://doi.org/10.1145/3325480.3329176
[13] Aubrey Colter, Patlapa Davivonga, Donald Derek Haddad, Halla Moore, Brian Tice, and Hiroshi Ishii. 2016. SoundFORMS: Manipulating Sound Through Touch. In *Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems* (San Jose, California, USA, May 7-12, 2016) (CHI '16). Association for Computing Machinery, New York, NY, USA, 2425-2430. https://doi.org/10.1145/2851581.2892414

[14] Maxime Cordeil, Benjamin Bach, Andrew Cunningham, Bastian Montoya, Ross T. Smith, Bruce H. Thomas, and Tim Dwyer. 2020. Embodied Axes: Tangible, Actuated Interaction for 3D Augmented Reality Data Spaces. In *Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems* (Honolulu, Hawaii, USA, April 25-30, 2020) (CHI '20). Association for Computing Machinery, New York, NY, USA, Article 486, 12 pages. https://doi.org/10.1145/3313831.3376613

[15] Céline Coutrix and Laurence Nigay. 2006. Mixed Reality: A Model of Mixed Interaction. In *Proceedings of the Working Conference on Advanced Visual Interfaces* (Venezia, Italy, May 23-26, 2006) (AVI '06). Association for Computing Machinery, New York, NY, USA, 43-50. https://doi.org/10.1145/1133265.1133274

[16] Nadine Couture, Guillaume Rivière, and Patrick Reuter. 2008. GeoTUI: A Tangible User Interface for Geoscience. In *Proceedings of the 2nd International Conference on Tangible and Embedded Interaction* (Bonn, Germany, February 18-20, 2008) (TEI '08). Association for Computing Machinery, New York, NY, USA, 89-96. https://doi.org/10.1145/1347390.1347411

[17] Andrew Dahley, Craig Wisneski, and Hiroshi Ishii. 1998. Water Lamp and Pinwheels: Ambient Projection of Digital Information into Architectural Space. In *CHI '98 Conference Summary on Human Factors in Computing Systems* (Los Angeles, California, USA, April 18-23, 1998) (CHI '98). Association for Computing Machinery, New York, NY, USA, 269-270. https://doi.org/10.1145/286498.286750

[18] Maxime Daniel, Guillaume Rivière, and Nadine Couture. 2019. CairnFORM: A Shape-Changing Ring Chart Notifying Renewable Energy Availability in Peripheral Locations. In *Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction* (Tempe, Arizona, USA, March 17-20, 2019) (TEI '19). Association for Computing Machinery, New York, NY, USA, 275-286. https://doi.org/10.1145/3294109.3295634

[19] Sarah Diefenbach, Eva Lenz, and Marc Hassenzahl. 2013. An Interaction Vocabulary. Describing the How of Interaction. In *CHI '13 Extended Abstracts on Human Factors in Computing Systems* (Paris, France, April 27 - May 2, 2013) (CHI '13). Association for Computing Machinery, New York, NY, USA, 607-612. https://doi.org/10.1145/2468356.2468463

[20] Emmanuel Dubois, Philip Gray, and Laurence Nigay. 2003. ASUR++: Supporting the design of mobile mixed systems. *Interacting with Computers* 15, 4 (Aug. 2003), 497-520. https://doi.org/10.1016/S0953-5438(03)00037-7

[21] Emmanuel Dubois, Philip D. Gray, and Laurence Nigay. 2002. ASUR++: A Design Notation for Mobile Mixed Systems. In *Proceedings of the 4th International Symposium on Mobile Human-Computer Interaction* (Pisa, Italy, September 18-20, 2002) (Mobile HCI '02). Springer-Verlag, Berlin, Heidelberg, 123-139. https://doi.org/10.5555/645739.666588

[22] Emmanuel Dubois, Paulo Pinheiro da Silva, and Philip Gray. 2002. Notational Support for the Design of Augmented Reality Systems. In *9th International Workshop on Interactive Systems: Design, Specification, and Verification* (Rostock, Germany, June 12-14, 2002) (DSV-IS 2002), Peter Forbrich, Quentin Limbourg, Jean Vanderdonckt, and Bodo Urban (Eds.). Springer Berlin Heidelberg, Berlin/Heidelberg, Germany, 74-88. https://doi.org/10.1007/3-540-36235-5-6

[23] Douglas C. Engelbart and William K. English. 1968. A Research Center for Augmenting Human Intellect. In *Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I* (San Francisco, California, USA, December 9-11, 1968) (*AFIPS '68* (Fall, part I)). Association for Computing Machinery, New York, NY, USA, 395-410. https://doi.org/10.1145/1476589.1476645

[24] Kenneth P. Fishkin. 2004. A Taxonomy for and Analysis of Tangible Interfaces. *Personal Ubiquitous Comput.* 8, 5 (Sept. 2004), 347-358. https://doi.org/10.1007/s00779-004-0297-4

[25] Kenneth P. Fishkin, Thomas P. Moran, and Beverly L. Harrison. 1999. Embodied User Interfaces: Towards Invisible User Interfaces. In *Engineering for Human-Computer Interaction: IFIP TC2/TC13 WG2.7/WG13.4 Seventh Working Conference on Engineering for Human-Computer Interaction* (Heraklion, Crete, Greece, September 14-18, 1998) (*EHCI 1998*). Stéphane Chatty and Prasun Dewan (Eds.). Springer US, Boston, Massachusetts, USA, 1-18. https://doi.org/10.1007/978-0-387-35349-4_1

[26] George W. Fitzmaurice, Hiroshi Ishii, and William A. S. Buxton. 1995. Bricks: Laying the Foundations for Graspable User Interfaces. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (Denver, Colorado, USA, May 7-11, 1995) (CHI '95). ACM Press/Addison-Wesley Publishing Co., USA, 442-449. https://doi.org/10.1145/223904.223964

[27] Morten Fjeld, Jonas Fredriksson, Martin Ejdestig, Florin Duca, Kristina Bötschi, Benedikt Voegli, and Patrick Juchli. 2007. Tangible User Interface for Chemistry Education: Comparative Evaluation and Re-Design. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (San Jose, California, USA, April 27 - May 3, 2007) (CHI '07). Association for Computing Machinery, New York, NY, USA, 805-808. https://doi.org/10.1145/1240624.1240745

[28] Morten Fjeld, Nicholas Ironmonger, Fred Voorhorst, Martin Bichsel, and Matthias Rauterberg. 1999. Camera control in a planar, graspable interface. In *Proceedings of the 17th IASTED International Conference on Applied Informatics (Innsbruck, Austria, February 15-18, 1999) (AI'99)*. IASTED/ACTA Press, Anaheim, California, USA, 242-245. https://doi.org/10.3929/ethz-a-010042243

[29] Stéphanie Fleck, Guillaume Rivière, Regina Ticona-Herrera, and Nadine Couture. 2018. Classifier les interfaces tangibles par la cladistique : critères et fiches pour un inventaire collaboratif [Classifying Tangible User Interfaces with Cladistics: Criteria and Forms for a Collaborative Inventory]. In *Proceedings of the 30th Conference on l'Interaction Homme-Machine* (Brest, France, October 23-26, 2018) (IHM '18). Association for Computing Machinery & AFIHM, New York, NY, USA & Grenoble, France, 209-218. https://doi.org/10.1145/3286689.3286705

[30] Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge, and Hiroshi Ishii. 2013. inFORM: Dynamic Physical Affordances and Constraints through Shape and Object Actuation. In *Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology* (St. Andrews, Scotland, United Kingdom, October 8-11, 2013) (UIST '13). Association for Computing Machinery, New York, NY, USA, 417-426. https://doi.org/10.1145/2501988.2502032

[31] John H. Frazer and Peter A. Frazer. 1980. Intelligent Physical Three-Dimensional Modelling Systems. In *Proceedings of Computer Graphics' 80* (Brighton, England, United Kingdom, August 13-15, 1980) (CG '80). Online Publications, Northwood Hills, Middlesex, United Kingdom, 359-370.

[32] John H. Frazer and Peter A. Frazer. 1982. Three-Dimensional Data Input Devices. In *Proceedings of Computer Graphics in the Building Process* (Washington, District of Columbia, USA, March 22-26, 1982) (BP '82). National Academy of Sciences, Washington, District of Columbia, USA, 409-416.

[33] Jérémy Frey, Renaud Gervais, Stéphanie Fleck, Fabien Lotte, and Martin Hachet. 2014. Teegi: Tangible EEG Interface. In *Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology* (Honolulu, Hawaii, USA, October 5-8, 2014) (UIST '14). Association for Computing Machinery, New York, NY, USA, 301-308. https://doi.org/10.1145/2642918.2647368

[34] Beverly L. Harrison, Kenneth P. Fishkin, Anuj Gujar, Carlos Mochon, and Roy Want. 1998. Squeeze Me, Hold Me, Tilt Me! An Exploration of Manipulative User Interfaces. In *Proceedings of the SIGCHI Conference on Human Factors in Computing Systems* (Los Angeles, California, USA, April 18-23, 1998) (CHI '98). ACM Press/Addison-Wesley Publishing Co., New York, 17-24. https://doi.org/10.1145/274644.274647

[35] Michael A. Hiltzik. 2000. *Dealers of Lightning: Xenon PARC and the Dawn of the Computer Age*. Harper Business, New York, NY, USA.

[36] Ken Hinckley, Robert J. K. Jacob, and Colin Ware. 2004. Input/output Devices and Interaction Techniques. In *The Computer Science Handbook* (2nd ed.), Allen B. Tucker (Ed.). Chapman and Hall/CRC Press, Boca Raton, Florida, USA, Chapter 20, 20.1-20.32.

[37] Ken Hinckley, Randy Pausch, John C. Goble, and Neal F. Kassell. 1994. Passive Real-World Interface Props for Neurosurgical Visualization. In *Conference Companion on Human Factors in Computing Systems* (Boston, Massachusetts, USA, April 2-7, 1994) (CHI '94). Association for Computing Machinery, New York, NY, USA, 452-458. https://doi.org/10.1145/259963.260443

[38] Lars Erik Holmquist. 2019. The Future of Tangible User Interfaces. *Interactions* 26, 5 (Aug. 2019), 82-85. https://doi.org/10.1145/3352157

[39] Lars Erik Holmquist. 2023. Bits Are Cheap, Atoms Are Expensive: Critiquing the Turn Towards Tangibility in HCI. In *Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems* (Hamburg, Germany, April 23-28, 2023) (CHI EA '23). Association for Computing Machinery, New York, NY, USA, Article 409, 8 pages. https://doi.org/10.1145/3544549.3582744

[40] Lars Erik Holmquist, Johan Redström, and Peter Ljungstrand. 1999. Token-Based Access to Digital Information. In *Handheld and Ubiquitous Computing: Proceedings of the First International Symposium* (Karlsruhe, Germany, September 27-29, 1999) (*HUC '99*), Hans-Werner Gellersen (Ed.). Springer Berlin Heidelberg, Germany, 234-245. https://doi.org/10.1007/3-540-48157-5_22

[41] Eva Hornecker, Trevor Hogan, Utä Hinrichs, and Rosa van Koningsbruggen. 2023. A Design Vocabulary for Data Physicalization. *ACM Trans. Comput.-Hum. Interact.* 31, Article 2 (Aug. 2023), 62 pages. Issue 1. https://doi.org/10.1145/3617366

[42] Elise van den Hoven and Berry Eggen. 2004. Tangible Computing in Everyday Life: Extending Current Frameworks for Tangible User Interfaces with Personal Objects. In *Ambient Intelligence: Second European Symposium, EUSAI 2004, Proceedings* (Eindhoven, The Netherlands, November 8-11, 2004) (*Lecture Notes in Computer Science*, Vol. 3295), Panos Markopoulos, Berry Eggen, Emile Aarts, and James L. Crowley (Eds.). Springer Berlin Heidelberg, Germany, 230-242. https://doi.org/10.1007/978-3-540-30473-9_22

[43] Hiroyasu Ichida, Yuichi Itoh, Yoshifumi Kitamura, and Fumio Kishino. 2004. Interactive Retrieval of 3D Shape Models Using Physical Objects. In *Proceedings of the 12th Annual ACM International Conference on Multimedia* (New York, NY, USA, October 10-16, 2004) (*MULTIMEDIA '04*). Association for Computing Machinery, New York, NY, USA, 692-699. https://doi.org/10.1145/1027527.1027685
[44] Hiroshi Ishii. 2008. Tangible Bits: Beyond Pixels. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (Bonn, Germany, February 18-20, 2008) (TEI '08). Association for Computing Machinery, New York, NY, USA, xv-xxv. https://doi.org/10.1145/1347390.1347392

[45] Hiroshi Ishii, Eran Ben-Joseph, John Underkoffler, Luke Yeung, Dan Chak, Zahra Kanji, and Ben Piper. 2002. Augmented Urban Planning Workbench: Overlaying Drawings, Physical Models and Digital Simulation. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality (Darmstadt, Germany, September 30 - October 1, 2002) (ISMAR '02). IEEE Computer Society, USA, 203-211. https://doi.org/10.1109/ISMAR.2002.1115090

[46] Hiroshi Ishii, H. Rich Fletcher, Jay Lee, Seungho Choo, Joanna Maria Berzowska, Craig Wisneski, Charlie Cano, Andres Hernandez, and Colin Bulthaup. 1999. musicBottles. In ACM SIGGRAPH 99 Conference Abstracts and Applications (Los Angeles, California, USA, August 8-13, 1999) (SIGGRAPH '99). Association for Computing Machinery, New York, NY, USA, 172. https://doi.org/10.1145/311625.311976

[47] Hiroshi Ishii, Dávid Lakatos, Leonardo Bonanni, and Jean-Baptiste Labrune. 2012. Radical Atoms: Beyond Tangible Bits, toward Transformable Materials. Interactions 19, 1 (Jan. 2012), 38-51. https://doi.org/10.1145/2065327.2065337

[48] Hiroshi Ishii, Ali Mazalek, and Jay Lee. 2001. Bottles as a Minimal Interface to Access Digital Information. In CHI '01 Extended Abstracts on Human Factors in Computing Systems (Seattle, Washington, USA, March 31 - April 5, 2001) (CHI '01). Association for Computing Machinery, New York, NY, USA, 187-188. https://doi.org/10.1145/634067.634180

[49] Hiroshi Ishii, Carlo Ratti, Ben Piper, Yao Wang, Assaf Biderman, and Eran Ben-Joseph. 2004. Bringing Clay and Sand into Digital Design — Continuous Tangible User Interfaces. BT Technology Journal 22, 4 (Oct. 2004), 287-299. https://doi.org/10.1023/B:BTJ].0000047607.16164.16

[50] Hiroshi Ishii, Sandia Ren, and Phil Frei. 2001. Pinwheels: Visualizing Information Flow in an Architectural Space. In CHI '01 Extended Abstracts on Human Factors in Computing Systems (Seattle, Washington, USA, March 31 - April 5, 2001) (CHI '01). Association for Computing Machinery, New York, NY, USA, 111-112. https://doi.org/10.1145/634067.634135

[51] Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA, March 22-27, 1997) (CHI '97). Association for Computing Machinery, New York, NY, USA, 234-241. https://doi.org/10.1145/258549.258715

[52] Hiroshi Ishii, Craig Wisneski, Scott Brave, Andrew Dahley, Matt Gorbet, Brygg Ullmer, and Paul Yarin. 1998. ambientROOM: Integrating Ambient Media with Architectural Space. In CHI '98 Conference Summary on Human Factors in Computing Systems (Los Angeles, California, USA, April 18-23, 1998) (CHI '98). Association for Computing Machinery, New York, NY, USA, 173-174. https://doi.org/10.1145/286498.286652

[53] Robert J. K. Jacob, Hiroshi Ishii, Gian Pangaro, and James Patten. 2002. A Tangible Interface for Organizing Information Using a Grid. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis, Minnesota, USA, April 20-25, 2002) (CHI '02). Association for Computing Machinery, New York, NY, USA, 339-346. https://doi.org/10.1145/503376.503437

[54] Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæk. 2015. Opportunities and Challenges for Data Physicalization. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea, April 18-23, 2015) (CHI '15). Association for Computing Machinery, New York, NY, USA, 3227-3236. https://doi.org/10.1145/2702123.2702180

[55] Ophélie Jobert, Thibaut Leone, Alix Goguey, Bruno Berberian, Julien Bourgeois, and Céline Coutrix. 2025. Effect of Robotic Modular Interface Assistance Type on Sense of Agency. In Proceedings of the 36th Conference on l'Interaction Humain-Machine (Toulouse, France, November 3-7, 2025) (IHM '25). Association for Computing Machinery & AFIHM, New York, NY, USA & Grenoble, France, Article 13, 18 pages. https://doi.org/10.1145/3765712.3765725

[56] Sergi Jordà, Günter Geiger, Marcos Alonso, and Martin Kaltenbrunner. 2007. The reacTable: Exploring the Synergy between Live Music Performance and Tabletop Tangible Interfaces. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (Baton Rouge, Louisiana, USA, February 15-17, 2007) (TEI '07). Association for Computing Machinery, New York, NY, USA, 139-146. https://doi.org/10.1145/1226969.1226998

[57] Hyunyoung Kim, Céline Coutrix, and Anne Roudaut. 2018. KnobSlider: Design of a Shape-Changing UI for Parameter Control. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal, Quebec, Canada, April 21-26, 2018) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3173574.3173913

[58] Hyunyoung Kim, Patricia Deud Guimarães, Céline Coutrix, and Anne Roudaut. 2019. ExpanDial: Designing a Shape-Changing Dial. In Proceedings of the 2019 on Designing Interactive Systems Conference (San Diego, California, USA, June 23-28, 2019) (DIS '19). Association for Computing Machinery, New York, NY, USA, 949-961. https://doi.org/10.1145/3322276.3322823

[59] Kazue Kobayashi, Mitsunori Hirano, Atsunobu Narita, and Hiroshi Ishii. 2003. A Tangible Interface for IP Network Simulation. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 5-10, 2003) (CHI EA '03). Association for Computing Machinery, New York, NY, USA, 800-801. https://doi.org/10.1145/765891.766000

[60] Daniel Leithinger and Hiroshi Ishii. 2010. Relief: A Scalable Actuated Shape Display. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (Cambridge, Massachusetts, USA, January 25-27, 2010) (TEI '10). Association for Computing Machinery, New York, NY, USA, 221-222. https://doi.org/10.1145/1709886.1709928

[61] Daniel Leithinger, David Lakatos, Anthony DeVincenzi, Matthew Blackshaw, and Hiroshi Ishii. 2011. Direct and Gestural Interaction with Relief: A 2.5D Shape Display. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA, October 16-19, 2011) (UIST '11). Association for Computing Machinery, New York, NY, USA, 541-548. https://doi.org/10.1145/2047196.2047268

[62] Rong-Hao Liang, Liwei Chan, Hung-Yu Tseng, Han-Chih Kuo, Da-Yuan Huang, De-Nian Yang, and Bing-Yu Chen. 2014. GaussBricks: Magnetic Building Blocks for Constructive Tangible Interactions on Portable Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada, April 26 - May 1, 2014) (CHI '14). Association for Computing Machinery, New York, NY, USA, 3153-3162. https://doi.org/10.1145/2556288.2557105

[63] Pierre Mahieux, Romain Biannic, Sébastien Kubicki, and Ronan Quarrec. 2022. SABLIER: A Tangible Interactor to Navigate through Space and Time. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, Louisiana, USA, April 30 - May 5, 2022) (CHI '22). Association for Computing Machinery, New York, NY, USA, Article 503, 12 pages. https://doi.org/10.1145/3491102.3517567

[64] Ali Mazalek and Elise van den Hoven. 2009. Framing Tangible Interaction Frameworks. Artif. Intell. Eng. Des. Anal. Manuf. 23, 3 (Aug. 2009), 225-235. https://doi.org/10.1017/S0890060409000201

[65] Zara Mirmalek. 2017. Inspiring Innovation: On Low-Tech in High-Tech Development. Interactions 24, 4 (June 2017), 50-55. https://doi.org/10.1145/3085562

[66] Gian Pangaro, Dan Maynes-Aminzade, and Hiroshi Ishii. 2002. The Actuated Workbench: Computer-Controlled Actuation in Tabletop Tangible Interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (Paris, France, October 27-30, 2002) (UIST '02). Association for Computing Machinery, New York, NY, USA, 181-190. https://doi.org/10.1145/571985.572011

[67] James Patten and Hiroshi Ishii. 2007. Mechanical Constraints as Computational Constraints in Tabletop Tangible Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 27 - May 3, 2007) (CHI '07). Association for Computing Machinery, New York, NY, USA, 809-818. https://doi.org/10.1145/1240624.1240746

[68] James Patten, Ben Recht, and Hiroshi Ishii. 2002. Audiopad: A Tag-Based Interface for Musical Performance. In Proceedings of the 2002 Conference on New Interfaces for Musical Expression (Dublin, Ireland, May 24-26, 2002) (NIME '02). National University of Singapore, Singapore, SGP, 1-6. https://doi.org/10.5281/zenodo.1176458

[69] James Patten, Ben Recht, and Hiroshi Ishii. 2006. Interaction Techniques for Musical Performance with Tabletop Tangible Interfaces. In Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (Hollywood, California, USA, June 14-16, 2006) (ACE '06). Association for Computing Machinery, New York, NY, USA, 27-es. https://doi.org/10.1145/1178823.1178856

[70] Esben Warming Pedersen and Kasper Hornbæk. 2009. mixiTUI: A Tangible Sequencer for Electronic Live Performances. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (Cambridge, England, United Kingdom, February 16-18, 2009) (TEI '09). Association for Computing Machinery, New York, NY, USA, 223-230. https://doi.org/10.1145/1517664.1517713

[71] Esben Warming Pedersen and Kasper Hornbæk. 2011. Tangible Bots: Interaction with Active Tangibles in Tabletop Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vancouver, British Columbia, Canada, May 7-12, 2011) (CHI '11). Association for Computing Machinery, New York, NY, USA, 2975-2984. https://doi.org/10.1145/1978942.1979384

[72] Radia Perlman. 1976. Using Computer Technology to Provide a Creative Learning Environment for Preschool Children. AI Memo 360 (May 1976), 31 pages.

[73] Ben Piper, Carlo Ratti, and Hiroshi Ishii. 2002. Illuminating Clay: A 3-D Tangible Interface for Landscape Analysis. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Minneapolis, Minnesota, USA, April 20-25, 2002) (CHI '02). Association for Computing Machinery, New York, NY, USA, 355-362. https://doi.org/10.1145/503376.503439

[74] Ivan Poupyrev, Tatsushi Nashida, and Makoto Okabe. 2007. Actuation and Tangible User Interfaces: The Vaucanson Duck, Robots, and Shape Displays.
In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (Baton Rouge, Louisiana, USA, February 15-17, 2007) (TEI '07). Association for Computing Machinery, New York, NY, USA, 205-212. https://doi.org/10.1145/1226969.1227012

[75] Laura Pruszko, Céline Coutrix, Yann Laurillau, Benoit Piranda, and Julien Bourgeois. 2021. Molecular HCI: Structuring the Cross-Disciplinary Space of Modular Shape-Changing User Interfaces. Proc. ACM Hum.-Comput. Interact. 5, EICS, Article 211 (June 2021), 33 pages. https://doi.org/10.1145/3461733

[76] Laura Pruszko, Hongri Gu, Julien Bourgeois, Yann Laurillau, and Céline Coutrix. 2023. Modular Tangible User Interfaces: Impact of Module Shape and Bonding Strength on Interaction. In Proceedings of the Seventeenth International Conference on Tangible, Embedded, and Embodied Interaction (Warsaw, Poland, February 26 - March 1, 2023) (TEI '23). Association for Computing Machinery, New York, NY, USA, Article 1, 15 pages. https://doi.org/10.1145/3569009.3572731

[77] Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shape-Changing Interfaces: A Review of the Design Space and Open Research Questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA, May 5-10, 2012) (CHI '12). Association for Computing Machinery, New York, NY, USA, 735-744. https://doi.org/10.1145/2207676.2207781

[78] Majken K. Rasmussen, Giovanni M. Troiano, Marianne G. Petersen, Jakob G. Simonsen, and Kasper Hornbæk. 2016. Sketching Shape-Changing Interfaces: Exploring Vocabulary, Metaphors Use, and Affordances. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA, May 7-12, 2016) (CHI '16). Association for Computing Machinery, New York, NY, USA, 2740-2751. https://doi.org/10.1145/2858036.2858183

[79] Matthias Rauterberg, Martin Bichsel, Uwe Leonhardt, and Markus Meier. 1997. BUILD-IT: a computer vision-based interaction technique of a planning tool for construction and design. In IFIP TC13 International Conference on Human-Computer Interaction (Sydney, New South Wales, Australia, July 14-18, 1997) (INTERACT '97), Steve Howard, Judy Hammond, and Gitte Lindgaard (Eds.). Springer US, Boston, Massachusetts, USA, 587-588. https://doi.org/10.1007/978-0-387-35175-9_95

[80] Matthias Rauterberg, Morten Fjeld, Helmut Krueger, Martin Bichsel, Uwe Leonhardt, and Markus Meier. 1998. BUILD-IT: A Planning Tool for Construction and Design. In CHI '98 Conference Summary on Human Factors in Computing Systems (Los Angeles, California, USA, April 18-23, 1998) (CHI '98). Association for Computing Machinery, New York, NY, USA, 177-178. https://doi.org/10.1145/286498.286657

[81] Patrick Reuter, Guillaume Rivière, Nadine Couture, Nicolas Sorraing, Loïc Espinasse, and Robert Vergnieux. 2007. ArcheoTUI - a Tangible User Interface for the Virtual Reassembly of Fractured Archeological Objects. In Proceedings of the 8th International Conference on Virtual Reality, Archaeology and Intelligent Cultural Heritage (Brighton, England, United Kingdom, November 27-29, 2007) (VAST'07). Eurographics Association, Goslar, Germany, 15-22. https://doi.org/10.2312/VAST/VAST07/015-022

[82] Guillaume Rivière. 2009. Interaction Tangible sur Table Interactive : application aux géosciences [Tangible Interaction on Tabletops: Applied in Geoscience]. Ph. D. Dissertation. University of Bordeaux 1, Bordeaux, France. Advisor(s) Nadine Couture and Maylis Delest. NNT2009BOR13837.

[83] Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2004. I/O Brush: Drawing with Everyday Objects as Ink. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria, April 24-29, 2004) (CHI '04). Association for Computing Machinery, New York, NY, USA, 303-310. https://doi.org/10.1145/985692.985731

[84] Kimiko Ryokai, Stefan Marti, and Hiroshi Ishii. 2007. I/O Brush: Beyond Static Collages. In CHI '07 Extended Abstracts on Human Factors in Computing Systems (San Jose, California, USA, April 27 - May 3, 2007) (CHI EA '07). Association for Computing Machinery, New York, NY, USA, 1995-2000. https://doi.org/10.1145/1240866.1240938

[85] Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present, and Future Directions. Foundations and Trends® in Human-Computer Interaction 3, 1-2 (Jan. 2010), 4-137. https://doi.org/10.1561/110000026

[86] Orit Shaer, Nancy Leland, Eduardo H. Calvillo-Gámez, and Robert J. K. Jacob. 2004. The TAC Paradigm: Specifying Tangible User Interfaces. Personal Ubiquitous Comput. 8, 5 (Sept. 2004), 359-369. https://doi.org/10.1007/s00779-004-0298-3

[87] Ben Shneiderman. 1983. Direct Manipulation: A Step Beyond Programming Languages. IEEE Computer 16, 8 (Aug. 1983), 57-69. https://doi.org/10.1109/MC.1983.1654471

[88] David Canfield Smith, Charles Irby, Ralph Kimball, and Eric Harslem. 1982. The Star User Interface: An Overview. In Proceedings of the National Computer Conference (Houston, Texas, USA, June 7-10, 1982) (AFIPS '82). Association for Computing Machinery, New York, NY, USA, 515-528. https://doi.org/10.1145/1500774.1500840

[89] David Canfield Smith, Charles Irby, Ralph Kimball, William Lawrence Verplank, and Eric Harslem. 1987. Designing the Star User Interface. Morgan Kaufmann

Publishers Inc., San Francisco, California, USA, 653-661.

[90] Steve Sutphen, Ehud Sharlin, Benjamin Watson, and John H. Frazer. 2000. Reviving a Tangible Interface Affording 3D Spatial Interaction. In Proceedings of 11th Western Canadian Computer Graphics Symposium (Panorama, British Columbia, Canada, March 26-29, 2000) (SkiGraph'00). 155-166.

[91] John Tiab and Kasper Hornbæk. 2016. Understanding Affordance, System State, and Feedback in Shape-Changing Buttons. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA, May 7-12, 2016) (CHI '16). Association for Computing Machinery, New York, NY, USA, 2752-2763. https://doi.org/10.1145/2858036.2858350

[92] Brygg Ullmer. 2002. Tangible Interfaces for Manipulating Aggregates of Digital Information. Ph. D. Dissertation. Massachusetts Institute of Technology, Cambridge, Massachusetts, USA. Advisor(s) Hiroshi Ishii. AAI0804673.

[93] Brygg Ullmer and Hiroshi Ishii. 1999. mediaBlocks: Tangible Interfaces for Online Media. In CHI '99 Extended Abstracts on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA, May 15-20, 1999) (CHI EA '99). Association for Computing Machinery, New York, NY, USA, 31-32. https://doi.org/10.1145/632716.632739

[94] Brygg Ullmer and Hiroshi Ishii. 2000. Emerging Frameworks for Tangible User Interfaces. IBM Syst. J. 39, 3-4 (July 2000), 915-931. https://doi.org/10.1147/sj.393.0915

[95] Brygg Ullmer, Hiroshi Ishii, and Dylan Glas. 1998. mediaBlocks: Physical Containers, Transports, and Controls for Online Media. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (Orlando, Florida, USA, July 19-24, 1998) (SIGGRAPH '98). Association for Computing Machinery, New York, NY, USA, 379-386. https://doi.org/10.1145/280814.2809400

[96] Brygg Ullmer, Hiroshi Ishii, and Robert J. K. Jacob. 2005. Token+constraint Systems for Tangible Interaction with Digital Information. ACM Trans. Comput.-Hum. Interact. 12, 1 (March 2005), 81-118. https://doi.org/10.1145/1057237.1057242

[97] Brygg Ullmer, Rajesh Sankaran, Srikanth Jandhyala, Blake Tregre, Cornelius Toole, Karun Kallakuri, Christopher Laan, Matthew Hess, Farid Harhad, Urban Wiggins, and Shining Sun. 2008. Tangible Menus and Interaction Trays: Core Tangibles for Common Physical/Digital Activities. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (Bonn, Germany, February 18-20, 2008) (TEI '08). Association for Computing Machinery, New York, NY, USA, 209-212. https://doi.org/10.1145/1347390.1347436

[98] Brygg Ullmer, Orit Shaer, Ali Mazalek, and Caroline Hummels. 2022. Weaving Fire into Form: Aspirations for Tangible and Embodied Interaction (1st ed.). Vol. 44. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3544564

[99] John Underkoffler and Hiroshi Ishii. 1999. Urp: A Luminous-Tangible Workbench for Urban Planning and Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA, May 15-20, 1999) (CHI '99). Association for Computing Machinery, New York, NY, USA, 386-393. https://doi.org/10.1145/302979.303114

[100] Anke van Oosterhout and Eve Hoggan. 2020. Reshaping Interaction with Rotary Knobs: Combining Form, Feel and Function. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, The Netherlands, July 6-10, 2020) (DIS '20). Association for Computing Machinery, New York, NY, USA, 1973-1982. https://doi.org/10.1145/3357236.3395536

[101] Annemiek Veldhuis, Rong-Hao Liang, and Tilde Bekker. 2020. CoDa: Collaborative Data Interpretation Through an Interactive Tangible Scatterplot. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney, New South Wales, Australia, February 9-12, 2020) (TEI '20). Association for Computing Machinery, New York, NY, USA, 323-336. https://doi.org/10.1145/3374920.3374934

[102] Alexander Wiethoff, Robert Kowalski, and Andreas Butz. 2011. inTUIt: Simple Identification on Tangible User Interfaces. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (Funchal, Madeira, Portugal, January 22-26, 2011) (TEI '11). Association for Computing Machinery, New York, NY, USA, 201-204. https://doi.org/10.1145/1935701.1935741

[103] Craig Wisneski, Hiroshi Ishii, Andrew Dahley, Matthew G. Gorbet, Scott Brave, Brygg Ullmer, and Paul Yarin. 1998. Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information. In Proceedings of the First International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture (Darmstadt, Germany, February 25-26, 1998) (CoBuild '98). Springer-Verlag, Berlin/Heidelberg, Germany, 22-32. https://doi.org/10.5555/645968.674740

[104] Andy Wu, Sam Mendenhall, Jayraj Jog, Loring Scotty Hoag, and Ali Mazalek. 2012. A Nested API Structure to Simplify Cross-Device Communication. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (Kingston, Ontario, Canada, February 19-22, 2012) (TEI '12). Association for Computing Machinery, New York, NY, USA, 225-232. https://doi.org/10.1145/2148131.2148180

[105] Lilith Yu, Chenfeng Gao, David Wu, and Ken Nakagaki. 2023. AeroRigUI: Actuated TUIs for Spatial Interaction Using Rigging Swarm Robots on Ceilings in Everyday Space. In Proceedings of the 2023 CHI Conference on Human
*Factors in Computing Systems* (Hamburg, Germany, April 23-28, 2023) (CHI '23).
Association for Computing Machinery, New York, NY, USA, Article 235, 18 pages.
https://doi.org/10.1145/3544548.3581437

[106] Clement Zheng, Jeeeun Kim, Daniel Leithinger, Mark D. Gross, and Ellen Yi-Luen Do. 2019. Mechamagnets: Designing and Fabricating Haptic and Functional Physical Inputs with Embedded Magnets. In *Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction* (Tempe, Arizona, USA, March 17-20, 2019) (TEI '19). Association for Computing

Machinery, New York, NY, USA, 325-334. https://doi.org/10.1145/3294109.3295622

[107] Jamie Zigelbaum, Adam Kumpf, Alejandro Vazquez, and Hiroshi Ishii. 2008. Slurp: Tangibility Spatiality and an Eyedropper. In *CHI '08 Extended Abstracts on Human Factors in Computing Systems* (Florence, Italy, May 5-10, 2008) (CHI EA '08). Association for Computing Machinery, New York, NY, USA, 2565-2574. https://doi.org/10.1145/1358628.1358713