Dancing Delicacies: Designing Computational Food for Dynamic
Dining Trajectories
Jialin Deng

Exertion Games Lab, Department of
Human-Centered Computing,Monash University, Melbourne,
Australia

Urs Dominic

Gaudenz Gaudi Labs LLC, Lucerne,
Switzerland

Humphrey Yang

Human-Computer Interaction
Institute, Carnegie Mellon University,
Pittsburgh, Pennsylvania, UnitedStatesLining Yao

Human-Computer Interaction
Institute, Carnegie Mellon University,
Pittsburgh, Pennsylvania, UnitedStatesAryan Saini

Exertion Games Lab, Department of
Human-Centred Computing, Monash
University, Melbourne, Victoria,
Australia

Patrick Olivier

Action Lab, Department of
Human-Centered Computing,Monash University, Melbourne, VIC,
Australia

Florian ‘Floyd’ Mueller

Exertion Games Lab, Department of
Human-Centered Computing,Monash University, Melbourne,
Australia

ABSTRACTKEYWORDS

Contemporary human-food interaction design is often a technologydriven endeavor in which food’s materiality has been largely underexplored. Building on the concept of “computational food”, thispaper explores the design of food as a material realization of computation through a material-centered approach. We engaged witha “Research through Design” exploration by designing a computational food system called “Dancing Delicacies”, which enables
food items to be “programmed” and “reconfigured” within dynamic
trajectories. Our practice led to a design framework resulting infour original dish designs. Our dishes aim to illustrate the richness of this new design space for computational food. Furthermore,through engaging with expert practitioners from the hospitality
industry, we provide a first account of understanding the design ofcomputational food for dynamic dining trajectories and its speculative use contexts in the industry. With this work, we hope to inspire
researchers and designers to envision a new future of human-foodinteraction.

Human-food interaction, Computational food design, Dynamic
trajectories

CCS CONCEPTS• Human-Centered Computing; • Interaction Design;

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for components of this work owned by others than theauthor(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from permissions@acm.org.
DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 978-1-4503-9893-0/23/07. . . $15.00
https://doi.org/10.1145/3563657.3596021

ACM Reference Format:
Jialin Deng, Humphrey Yang, Aryan Saini, Urs Dominic, Lining Yao, Patrick
Olivier, and Florian ‘Floyd’ Mueller. 2023. Dancing Delicacies: Designing
Computational Food for Dynamic Dining Trajectories. In Designing Interactive Systems Conference (DIS ’23), July 10–14, 2023, Pittsburgh, PA, USA. ACM,
New York, NY, USA, 18 pages. https://doi.org/10.1145/3563657.3596021

1

INTRODUCTION

Explorations in the HCI field have led to greater convergence between food and computing technology, giving rise to an emerging
research field: Human-Food Interaction (HFI) [7, 20, 21, 23, 26, 44].
However, the current research agenda in HFI appears to be primarily
driven by technological novelty and efficiency [24, 25], rather thanfood’s material affordances [29]. Interestingly, we noted that culinary practitioners have long been experimenting with food’s materiality by exploiting and manipulating food’s physical and chemical
properties [77], resulting in innovations in new cuisines and “magical” food experiences [74]. However, many HFI designs fall shortwhen it comes to “celebrat[ing] the pleasurable and enjoyable experiences that people have with food” [34], which are often derived
from the food material’s properties emphasizing the aesthetic, affective, sensual and sociocultural qualities of food [81, 82, 89].
Nevertheless, prior research has proposed an emerging notionof “computational food” and initially envisioned a future where
food as a material is computationally transformable and reconfigurable [8, 42]. Drawing on this vision, researchers proposed the“material integration” where food is seen as a material that mediates
the realization of computation [22, 25, 27]. This approach led tothe creation of a series of computational desserts that can react tosimple inputs with basic logic operations because of the integratedfunctionality within the desserts’ physical structures [25]. ThisDIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Figure 1: An exemplar dish of Dancing Delicacies.
work suggested that the changes in food’s states via computation
could enrich people’s engagements with food, yet only focused ona single “one-off” interaction for diners (akin to a button toggle).
Therefore, we aim to extend the prior work from a one-off configuration toward more temporal processes of food that could behave
dynamically in response to multiple operations.
In this paper, we engaged with a “Research through Design”(RtD) [95] process to investigate dynamic trajectories in food creation and consumption by creating a computational food systemcalled “Dancing Delicacies”. The system enables food items to be“programmed” and “reconfigured” through a series of dynamic operations (Section 4). Specifically, through “programming”, a foodcreator can define the setting of the starting conditions of a dishand its dynamic behaviors (i.e., moving trajectories), and throughconfiguring, a diner is offered several parameters to control the foodprocess according to the settings defined by the creator (e.g., letting
diners to change the food layout or combine different flavors). We
present a design framework encompassing two dimensions related
to dynamic trajectories when designing computational food: “levelof control” and “gastronomic value”. Furthermore, to explore how
our framework facilitates new ways of designing computational
food, we created four exemplar Dancing Delicacies dishes illustrating four distinctive design directions and their implications (Section
5). Lastly, we used our design to explore speculative use contexts
through a preliminary engagement with expert practitioners (N=5)
from the hospitality industry (Section 6).
Taken together, our contributions include:• A new system for computational food designs that facilitates dynamic dining trajectories via “programming” and“reconfiguring” food items.
• A design framework depicting a new space of computational
food that could serve as a basis for critically analyzing andunderstanding computational food designs.

• Four exemplar “Dancing Delicacies” dish designs suggesting
multiple design directions concerning future design possibilities of dynamic dining trajectories within computationalfood.
• Seven themes derived from a preliminary engagement withhospitality experts that provide a first account of understanding the design of computational food for dynamic dining
trajectories and its speculative use contexts in the industry.

2 RELATED WORK
2.1 Approaches to current HFI design
There has been a notable increase of works in HFI highlighting thepossibilities enabled by technology to impact our food practices andexperiences [7, 20, 21, 23, 26, 44]. HFI researchers have begun experimenting with emerging technologies such as 3D-printing [47, 76],
virtual reality [10], capacitance sensing [37, 87, 88], robotics [58],electrical muscle stimulation [65], acoustic levitation [83, 84], andshape-changing interfaces [66, 86]. These developments are supplemented by smart kitchens [59] and appliances [51, 76] that enableenhanced food preparation processes. At the same time, smartphone apps employ artificial intelligence to track users’ diets (e.g.,calorie consumption) by automatically identifying food items andcomputing the recommended volume and nutritional information[69].
We note that these existing HFI works are often a technologydriven endeavor highlighting the functionality and novelty of computing technology. However, such endeavors might only be one of
the many approaches through which food and technology design
could be equally engaged. Indeed, Comber et al. [19] argued that
HFI should pay much more attention to the people and how they
engage with food than the efficiencies and novelties new technologies may offer. Echoing this, we note that the inherent affordancesof food still appear to be overlooked in existing HFI approaches[6, 22-25].

Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

2.2

Engaging with food’s materiality for
dynamic presentations in contemporary
gastronomy

We find that practitioners in contemporary gastronomy have been
engaging with food’s materiality for the innovation of new cuisines
to enrich dining experiences. Specifically, chefs have developed“animated” food that changes its original state on the plate (e.g.,shape, color) to create “magical” food experiences [74]. For example,
the dessert “Flor de Cacao” [35] represents a cocoa bean that opens
up like a cacao flower through contact with hot chocolate sauce.
Similarly, mixologists have shown a growing interest in producing color-changing cocktails. One prominent example is a cocktail
with a pH-sensitive ingredient (butterfly pea flower tea) that can
instantly change from blue to violet when blended with citric drinks[70]. Such state changes in food realize a “dynamic presentation”
to please the diners. However, while culinary techniques like thesecan often facilitate a pleasurable dining experience, they offer only
one fixed way of enjoying the food/drink once it has been produced.
Such “one-off” experiences mean that the dining process is predominantly pre-set by the food creator (i.e., the chef), and, consequently,
there is little room for “inter-actions” [89] between the diners andthe food.
In response to this, we intend to explore the possibility of employing food’s material qualities to realize computational food ina more temporal way. In particular, we aim to use food’s physicalprocess for the direct realization of a computational process (e.g.,take in information, perform logical operations, sense, and react[71]), and simultaneously, use computational capabilities to achieve
desired operations and effects (e.g., shape/color changes) that candynamically change over time.

2.3

Learning from computational matters toachieve transformative qualities

Many forms of matter have complex structures and dynamics thatcan be regarded as a computational system, relating certain computational properties and powers to the material substrate’s physicalproperties [75]. Prior research explored various ways to program
highly deformable materials that can respond to various external
stimuli, including using synergetic chemical and physical programming for reversible shape-shifting actuators [52, 72] and biohybridmaterials in a smart wearable that can reversibly change its shapein response to body temperature and humidity [92]. Furthermore,
researchers have experimented with the programmability of foodmaterials, including the encoding of active structures into food materials, which enables the food to transform physically in response
to external stimuli [42, 86]. These examples show that there are avariety of possibilities to encode computational capabilities, suchas actuation or sensing, into certain materials. Nevertheless, thedesign of dynamic trajectories enabled by the computational poweris still underexplored.
We took inspiration from prior research on “material integration”[22] that introduced the notion of “food as computational artifact”.
Specifically, the authors designed a dessert that can realize the basic
building blocks of computation – the three basic logic gates (AND,OR, and XOR) – and “compute” its own flavor [25]. Moving beyond
this work, in this paper, we aim to extend the view of computational

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

food away from the design of a “thing” with limited transitions,
toward the design of food as “events” involving dynamic changes[90].
Taken together, our review of prior work yielded three major
insights: i) existing HFI approaches appear to have missed opportunities to engage with the affordances of food; ii) culinary practices
have already embraced the opportunity to engage with dynamic
food using its material affordances; iii) and, although research has
begun to explore food as computational material, prior work has
so far focused on limited physical transitions and missed opportunities for reconfigurations via dynamic trajectories. In response, inthis paper, we attempt to answer the research question: How do wedesign computational food for dynamic trajectories?

3

METHODOLOGY

We explored the dynamic trajectories within computational food
through a combination of “Research through Design” (RtD) [94, 95]
and “material speculation” [85]. In RtD, the design of a novel artifact,
as a reflective practice, is a source of new knowledge that is “topical,
procedural, pragmatic and conceptual” [31]. In this way, we see ourdesign not as a final product but as a “material speculation” [85]within the RtD tradition. We aim to exploit Dancing Delicacies to
provoke possible world accounts which extend the inquiry beyond
the system itself. In other words, we see our design as a proposition“being at the boundary of the actual and the possible”, to encourage
speculation and inspire possible HFI futures [28, 85].
In undertaking this exploration, we recognize that RtD is nota singular research method or approach, rather, it is “a multitudeof legitimate intersections” between design practice and research[57]. Specifically, in this paper, our RtD exploration has four facets.
Firstly, we documented our prototyping process of “Dancing Delicacies” system. This process was a means of inquiry (i.e., experimentation and reflection) into new ways of making and interactingwith computational food. Secondly, our designed system serves the
purpose of a “research archetype” [57], that is, a “physical embodiment” of a new design space [57] corresponding to our proposed
conception of computational food. It concretizes our latter concept of shifting relationships within novel creator-food-consumer
interactions enabled by dynamic trajectories with computational
food. Thirdly, we identify the key dimensions within a framework
to illustrate the richness of the emerging design directions of dynamic trajectories with computational food. Also, by demonstrating
four exemplar Dancing Delicacies dishes as “narrative proposals”[32] and positioning them within our framework, we intended to
explicate different design directions concerning future design possibilities. Finally, through a preliminary engagement with expert
practitioners from the hospitality industry, we leverage our designs
to provoke insights into speculative use contexts.

4 DESIGN PROCESS
4.1 Exploiting electrowetting technology forcomputational food design
Taking inspiration from the notion of “unconventional computing”,
which is grounded in the intention to conceive “other ways to compute” [3], we note that a computer can take alternative forms, suchas computers made of liquids [2]. Liquids, especially in the form of

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Figure 2: Concept sketches of Dancing Delicacies.
water droplets, are particularly versatile computational materials[43] that can be manipulated and programmed with evaporative[17], optical [9], acoustic [67], magnetic [53], and electric forces ona surface [33] and in a dielectric fluid [38]. Also, it is important tonote that liquids, such as soup, broth, coulis, liquor, and syrup, areessential elements in culinary practices, not only for gustatory enjoyment but also for the overall sense of aesthetic delight in eating
experiences.
In particular, our design was inspired by prior research that exploited “electrowetting on dielectric (EWOD)” technology [15, 56,
79, 80, 93] to achieve precise motion operations of aqueous droplets,and used them as an interaction medium without altering their material composition [80]. EWOD technology utilizes electrical voltage
to unbalance the force equilibrium at the solid-liquid-vapor interface, causing the droplets to move toward the charged electrodes[5]. Previously, EWOD systems have been used mostly in biologicaland medical experiments, such as for DNA extraction [4], and drugdelivery [64]. Other applications used EWOD technology for opticalimaging [36], power energy harvesting [49], and very rarely, foodanalyzing and testing [12], but not for culinary purposes. Furthermore, prior work suggested a set of strategies for droplet operations
with a non-food EWOD system, including “moving”, “dispensing”,“merging”, and “mixing” [78, 80]. We draw from this prior knowledge around EWOD and extend it to food experiences to realize ourdesign. Our design concept (Figure 2) is a dish on a plate containing
different food items and ingredients; some food items are in theform of aqueous droplets that can be manipulated using EWOD
technology embedded in the plate. A chef can “program” the dishduring the process of its creation and choreograph the diner’s encounters within the dining trajectories. Based on how the dish has
been programmed, the diner can “reconfigure” it to enjoy different
eating trajectories.
We found that most existing EWOD systems used a fixed form
with micro-sized electrodes. This means that the systems can only
handle small volumes of liquids, which makes it challenging toserve a sufficient portion of a proper dish. Furthermore, althougha few systems worked with foodstuff, they were not specificallydesigned for authentic dishes in a dining context. Therefore, in
our design, we first increased the size of a single electrode fromthe traditional millimeter scale to a centimeter scale (1cm*1cm),
enabling the effective manipulation of a droplet from 215𝜇L to

430𝜇L. We then arranged the electrode array into a particular shapeand embedded the electrode board into a 3D-printed plateware as
the container of food droplets. We detail our prototyping process
next.

4.2

Prototyping the Dancing Delicacies system

Our final design of the plateware has three major components: anelectrode board, a dielectric film surface that is held by a frame,
and a 3D-printed plate with two assembling parts (top and base)(Figure 3). Our design iteration process was primarily guided by
the following constraints and considerations:• Technical constraints. We intended to utilize the OpenDrop
V4 controller [5] to realize our design. Therefore, we wereconstrained by the maximum number of electrodes the controller can support: 128.
• Utilitarian considerations: We arranged the electrodes in acircular shape to ensure the overall size of the plateware isin line with regular dish plates (∅:230mm) while zoning theplateware into two different functional divisions (Figure 3).
The fabrication process included customizing the electrode board,
making a dielectric surface, and designing a plateware. This wasfollowed by combining all components into a visually pleasing
design.
We created an electrode board by manufacturing a multi-layer
printed circuit board (PCB). The electrode board comprises of asilver-coated 128 electrodes array with each individually 10 mm x
10 mm in size and 0.127mm gaps between electrodes (Figure 4a-b).
There are also two HV507 drivers underneath the board (Figure 4c).
The electrode board interfaces with a controller to realize droplet
manipulation. The successful movement of a droplet from one
electrode to another depends upon the reversible polarization ofa surface made of a thin dielectric film, which sits on top of theelectrode board. The performance of EWOD is highly dependent on
the thickness and surface energy of the dielectric film [5, 78, 80], andwe have used “Parafilm®M laboratory film” as recommended [5, 78].
The electrode board and dielectric film were assembled within a
3-D printed plate and connected to an OpenDrop V4 controller that
operates at 240V DC and 1000Hz via an FFC cable (Figure 6). Notethat the dielectric film electrically insulates the electrode board andDancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

Figure 3: Functional divisions (top) and schematic structure(bottom) of the plateware.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

represents the results of an initial investigation of readily available
food materials with different properties.
Prior work with non-food droplets [56] demonstrated that thehysteresis of a droplet (which causes some drops to stick on the surface) has been one of the shortcomings of EWOD technology. There
has been limited exploration of the various properties of foods andbeverages that might cause hysteresis when using EWOD technology. We hence began with off-the-shelf aqueous food materials
including water, tea, vinegar, soda, milk, liquor, coffee, and wine
which we determined could make suitable operational droplets (i.e.,the system could manipulate their movements). We found that aliquid’s viscosity might increase the probability of droplet hysteresis, confirming prior work on non-food droplets [56]. This result
could be due to the viscous damping of the surface capillary waves,which leads to an increased inertial wetting time [16]. We also
experimented with solid, soluble, and insoluble food items, such as
small pieces of bread, herbs, tea leaves, instant coffee, and Vitamin
C effervescent tablets. We found that these food items can be used
to alter the operational droplets’ flavors and their states by soaking,infusing, and dissolving them in the droplets. It is worth noting thatthe buoyancy of a solid item (whether the item can be transported
by a droplet) is determined by multiphysics (e.g., density, wettability, aspect ratio, etc.). Lastly, inspired by molecular gastronomy,
we also found that compounds, such as sodium alginate, calcium
chloride, gelatin, and agar-agar, could be added to the droplets tochange their properties through physical and chemical reactions.

4.4

Figure 4: a) A printed circuit board holds the electrode array;b-c) The individual electrodes are connected to the backsideof the circuit board through vertical interconnects and linked
to a connector on the back with 128 channels.

Figure 5: a) Fabricating the electrode board; b) Making adielectric film surface held by a 3D-printed frame; c) 3Dprinting a plate case.

the charge on the electrodes from the system delivered to a dineris minimal [78].

4.3

Engaging with food materials

We identified a library of food materials that can best engage withthe system (Figure 7). Note that this library is not exhaustive; it

Fundamental droplet operations

We offer a “visual interface” as a design tool for programming food.
This tool is built upon the original work by OpenDrop and offersa simple and visual way allowing food creators, such as chefs, tomanipulate the food droplets’ movements in real-time. Throughcontrolling electrode on/off states frame by frame, chefs can realize
desired operations of food items and choreograph various dining
trajectories that the diners would encounter (Figure 8).
We explored a set of fundamental droplet operations by using thevisual interface. Inspired by prior works with a non-food EWOD
system [78, 80], we identified eight fundamental droplet operations
that our system makes possible, and we categorized them into fourlevels according to complexity (Figure 9).
The base level (lowest complexity) encompasses two basic “move”
operations: travel and merge. Travel refers to one or multiple
droplets changing their original location, moving from one cellto another following a predefined path. The maximum droplet(volume=200𝜇L) velocity with our system was 17.5mm/s. Merge
refers to two or more droplets moving toward each other and meeting at the same electrode.
Level one, building on the base level, encompasses two more
complex operations: oscillate and blend. Oscillate means a droplet
keeps moving from one electrode to another and back again atregular temporal intervals. For example, a droplet keeps moving
back and forth between points A and B. Blend refers to two or more
droplets meeting at some point and blending into a mixture. For
example, two droplets meet and circle from point A to B to C to D
and back to A to blend with each other until evenly mixed.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Figure 6: A complete Dancing Delicacies system consists of a plateware with an embedded electrode board connected to an
OpenDrop V4 controller via an FFC cable, and a USB-C cable for communication and powering.

Figure 7: Food materials and their utility.
Level two encompasses operations that involve extra (solid) fooditems apart from droplets: delivery, and transport. Delivery is concerned with one or more droplets carrying additional food items to acertain destination. For example, in our test, a droplet carries a driedbasil leaf fragment from point A to destination B (distance=60mm,
time=9s). At destination B, the droplet “drops” the leaf as the droplet
is soaked up by, for example, a piece of bread. Transport refers toa droplet carrying additional food items moving around the plate.
For example, a droplet carries a leaf while moving along a predefined pathway. This operation could be useful in situations whenfood items should be moved, but the movement is not supportedby the underlying technology (i.e., leaves do not respond to the
EWOD technology). To transport larger items, we could increase
the volume and hence enlarge the cargo capacity of the droplet.
Level three considers additional consequences caused by thebase operations: reaction and release. Reaction refers to droplets
moving so that they physically or chemically react with other items.
For example, if a droplet meets an effervescent tablet, it begins tobubble up. Release means that the water-soluble item gradually
dissolves in the droplet while it is moving around. For example, apiece of freeze-dried instant coffee is slowly released into a droplet.

Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Figure 8: The visual interface for programming food.

4.5

Possible interactions with Dancing
Delicacies

Next, we describe possible interactions with Dancing Delicacies.
We were inspired by prior non-food EWOD work that identified
various ways of moving droplets, including directly touching thedroplets, tilting the interaction surface, blowing onto the droplets,
and manually adding and removing the droplets using pipettes orwiping them off [78, 80]. Building on this, we identified three approaches: direct manipulation, time-based interaction, and sensing.
“Direct manipulation”, inspired by the general HCI term, refers
to the diner changing the droplet’s movements through physical actions. For example, the diner could use a fingertip or cutlery to dragthe droplet to a desired location; add food items onto or remove
them from electrodes to block or unblock the path of a moving
droplet; blow onto droplets; or tilt the plate to change the directionof the droplets’ movement. While direct manipulation could potentially encourage a close personal engagement, it only allows thediner to act on one droplet or a small number of adjacent dropletsat a time. This approach could also cause accidental activations(e.g., droplet collision and unintended interactions).
“Time-based interaction” refers to exploiting “time” as a design
source for interactions with Dancing Delicacies to coordinate multiple operations in real-time. For example, a diner might need to wait
for a droplet’s movement to be carried out within a pre-programmed
amount of time before they can carry out a new action. Time-based
interaction allows for synchronization between diners’ actions andcomputational operations (directed by the food creator).
Lastly, while we have not yet implemented it in our system, thecapability of sensing has been explored with EWOD devices [40, 54],
using embedded capacitive sensing units to realize the detection
and reading of droplets’ locations and movements, and to recognize
the presence of other food items (mostly conductive), and to process
any feedback obtained. Based on this understanding, we propose
possible dish designs that might exploit this approach (section 5.3).

Figure 9: Fundamental droplet operations.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

5

DESIGN SPACE

We developed a two-dimensional framework illustrating an emerging design space of computational food with dynamic trajectories.
The framework saves as a conceptual constraint that can be utilized
by designers to reflect on the distinctiveness between different designs. It might also help designers identify underexplored areas,highlighting opportunities for future investigations. In this section,
we first articulate the two dimensions within our design space (5.1);
then we present four design directions emerging from the four
quadrants, demonstrated through four design exemplars of Dancing Delicacies dishes (5.2); lastly, we discuss the implications ofdynamic trajectories for HFI through a reflection on the designed
exemplar dishes (5.3).

5.1

Two dimensions

We identified two dimensions that designers might take into consideration when aiming to design dynamic trajectories within computational food (Figure 10): “level of the diner’s control over thedish’s computational processes”, and “gastronomic value”. The first
dimension was derived from the inherent shifting of control derived
through the interactions with the food facilitated by computational
processes that we identified through our design practice, as suggested in prior work [27]. The second dimension was inspired byprior findings that design intentions can be directed by sensations
and perceptions – where interaction designers can either focus ontechnology that provides “localized sensations” (e.g., tactility andtaste) or “perceptions outside of myself” (e.g., vision) [60] – here,
we have identified two ends through our design practice that point
at two distinct design intentions: “gustatory-focused” (taste stimuli)and “aesthetic-focused” (visual stimuli).
The first dimension, plotted on the horizontal axis, indicates thelevel of the diner’s control over the dish’s computational process
when eating, ranging from a low to a high level. A low level of

Jialin Deng et al.

control means that the dish affords fewer parameters for diners
to control the dish’s computation. This could provide sufficientguidance to lead diners into predefined trajectories to enjoy thedish. In contrast, a high level of control means that the diner has
more parameters to control the dish. Higher levels of control areusually implemented to give the diner more agency to personalize
their dish and explore unknown flavor experiences.
The second dimension, plotted on the vertical axis, denotes thegastronomic value of a dish as aimed for by the food creator throughthe computational process. Gustatory-focused refers to a design
intention that is more inclined to focus on the transitioning ofa dish’s tastes and textures (mouthfeel) that provides diners withpalatable “localized sensations”. Aesthetic-focused refers to a design
intention that is more inclined to focus on the transitioning ofa dish’s visual appearances and movements that provides diners
with varying “outside of myself” perceptions. Note that althoughfood creators could aim to balance a dish’s palatability and visualpresentation, we find that there often appears to be a compromisethat they need to make (usually due to technical limitations). We
now describe four quadrants that result from the two dimensions,which help to explain the different options food creators have.

5.2

Design directions with exemplar Dancing
Delicacies dishes

The dimensions form four quadrants that allow us to articulate four
design directions: “instructional”, “directional”, “performative” and“freeform” (Figure 10). We hope that these design directions could
benefit food creators by providing a starting point on which to orient their decisions, establish their design goals, as well as envisage
the possible outcomes. We articulate each quadrant through ourdesign of four exemplar Dancing Delicacies dishes. These exemplar dishes serve as “narrative proposals” which remain “open to
imaginary extensions, developments, and modifications” for future

Figure 10: Design space of dynamic trajectories within computational food.

Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Figure 11: Initial plating of Jelly Wagashi.

Figure 12: Flow chart of interacting with Jelly Wagashi.
design possibilities [32]. As such, the dish designs are “placeholders” [32], occupying each direction in the design space without
necessarily being the perfect designs to populate it.

the droplet is fully mixed after five rounds of circling, it travels to
the main food item, the plain jelly cake, which soaks up the droplet(KF5) and becomes flavorful and ready for consumption.

5.2.1 Instructional dish: Jelly Wagashi. The dish “Jelly Wagashi” issituated in the upper left quadrant which is more gustatory-focused,and diners have a low level of control over the computation. It
facilitates an instructional trajectory of eating through engaging
with a certain recipe and dining process that was predefined by thefood creator. Diners mostly follow the lead provided by the dishbased on the food creator’s programming. The plate is divided into
four different functional areas (Figure 11). While the dish follows apredefined series of steps to prepare the meal as the droplet moves
through different areas, the diner is given the option to nudge
flavorings or additional ingredients to the droplet to alter its flavor.
There are five steps to enjoy the Jelly Wagashi dish. These areshown in the flowchart, with five keyframes (referred to as “KF”)(Figure 15). When the dish is served, the diner is presented with aflavoring droplet oscillating in the synthesizer area, suggesting thatthe diner uses their cutlery (here, a small spoon) to drive a base
droplet toward it and wait until the two droplets are fully mixed(KF2). Next, the mixed droplet travels toward the solid food item
repository. It stops for 5 seconds in front of each of the solid fooditems to allow the diner to put one or multiple solid food items
onto the droplet (KF3). Then, the droplet, carrying the food item(s),moves toward the beverage synthesizer area, where it blends witha beverage droplet via the circling “blend” operation (KF4). Once

5.2.2 Directional dish 2: Meta Platter. The “Meta Platter” dish sits
in the upper-right quadrant, it is gustatory-focused, and gives diners
a high level of control over the dish’s computation. This dish offers
a decision-based eating trajectory where the diner keeps coming
to decision points and must choose between options. Each choicealters the droplet’s trajectory and leads the diner down a different
path toward one of many possible outcomes (Figure 14). Based on
the diner’s choices, the controller acknowledges user input throughits sensing functions, then proceeds and moves the droplet down
the corresponding path until the droplet meets the next decision
point or a termination condition. As a result, food preparation andits possible outcomes are represented with a decision tree, where
each node is a decision point, and the terminal nodes symbolize
the possible food outcomes. In our design, there are four different
branching trajectories that a diner would encounter. Note this dishinvolves “sensing”, where we leave the implementation for future
research. In this paper, we simply demonstrate the snapshots using
the visual user interface (Figure 16).
Figure 15 shows the flow of one probable trajectory that a dinercould go through. The diner encounters the first branch at decision
point 1, where they must decide to nudge the base droplet towardfood item A or B. Assuming that the diner selects A, a mixture of C
as outcome 1 is generated, which moves toward decision point 2.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Figure 13: Demonstration of Jelly Wagashi in action. Our recipe: 1) Jelly cake (main food item): gelling powder (carrageenan),raspberry jam, sugar; 2) Droplets in operation (dyed with food colors): lemon juice (yellow), honey drink (blue), strawberry
juice (red); 3) Solid food items: green tea, mint leaf, coffee grains.

Figure 14: Initial plating of the Meta Platter with a branching traffic map with four different trajectories depending on the
diner’s choices.

Figure 15: Flow chart of interacting with Meta Platter.
Again, the diner needs to choose between the solid food items D or
E. If the diner selects E, the mixture C combines with E, producing F
as outcome 2. Finally, the result is blended with a beverage droplet
in the beverage synthesizer, then soaked up by the main food item,
resulting in the final outcome that is ready for consumption.
5.2.3 Performative dish 3: Seasonal Transience. The performative
dish “Seasonal Transience” sits in the lower-left quadrant, and
it is more concerned with the dish’s presentation (and less with
how it tastes). Dishes in this quadrant often afford a low levelof control over the computation while focusing on the aesthetics
that highlights the visual appeal and expressiveness of a dish. The

Seasonal Transience dish was inspired by Japanese sweets thatreflect the changing seasons [1, 30]. Using the plate as a stage
to present a performance, diners could step into an edible story
with four different trajectories that transform with the seasons.
The plate is divided into four areas for serving different foods
representing each season. Each area is populated with droplets
and additional food items as props. A mint leaf is carried on a
droplet and moves through the areas (seasons), indicating the flowof the performance (Figure 17). As the leaf moves toward an area, it
triggers the corresponding executions in each area simultaneously.

Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Figure 16: Demonstration of Meta Platter in action. Our recipe: 1) Main food item: aged cheddar, artisan crackers, almond; 2)
Optional droplets (some of them were dyed with food colors): lemon juice (yellow), vinegar (brown), strawberry juice (red), tea(light yellow), grapefruit juice (pink); 3) Solid food items: cabanossi with rosemary, cheese with coconut flakes, coffee grains,
dried basil leaves, green tea leaves.

Figure 17: Initial plating of Seasonal Transience.

Figure 18: Flow chart of interacting with Seasonal Transience.
The trajectory starts with the migratory mint leaf moving toward
the winter season (Figure 18), and simultaneously, two dropletsof flavoring (e.g., warm strawberry juice) move toward each otherand then mix with the food items (yogurt sorbet) in the winterarea (a). When the sorbet starts to melt, representing melting snow
in winter, diners can start to enjoy the first course in the winter.
After five minutes, the mint leaf begins to move again, toward thespring area. At the same time, two droplets in the area start tomove toward a “bud” (a flower garnish made of rice paper) (b). The
bud is soaked by the droplets and begins to open into a flower,
representing the spring bloom. Diners can now start to enjoy thesecond course in the spring (c). Next, the mint leaf moves toward
the summer after 5 minutes while two droplets move toward a food
item (e.g., a small piece of effervescent fruit tablet) in the summer
area. The tablet quickly reacts to the droplets, producing a fizzing
sound that mimics rainfall in the lush green summer (d). Finally,
the mint leaf’s trip ends at its last stop, the autumn. Two droplets
blend into a golden yellow color, representing a golden autumn (e).
5.2.4 Freeform dish: Tic Tac Toe. The “Tic Tac Toe” dish sits inthe lower-right quadrant. Dishes in this quadrant facilitate a highlevel of control over the computation, and their design tends toDIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Figure 19: Demonstration of Seasonal Transience in action. Our recipe: 1) Additional food item (center) and garnishes: ricecracker, rice paper cutout (flower), mint leaf; 2) Winter: strawberry juice (red), frozen yogurt; 2) Spring: green juice, flower
garnish (rice paper), goji berry; 3) Summer: tropical fruit juice, effervescent tablet (lemon); 4) Autumn: jasmine tea, orange
juice.

Figure 20: Initial plating of Tic Tac Toe.
be dominated by aesthetic considerations. Here, we propose an
edible game which is divided into a droplet repository and a 3-by-3
grid playground. It is programmed to play a tic tac toe game withthe diner. The playground is diagonally oriented to avoid droplet
adjacency. The droplet’s behaviors (i.e., rules and droplet paths) canbe directly programmed into the controller via an Arduino script[31], and the antagonistic moves are generated based on the stateof the board (Figure 20). As this dish involves using sensing andthe path-planning algorithm to process feedback obtained fromthe presence of food items and avoid droplet collision, we leave
the implementation of the dish for future research. In this paper,
we demonstrate the snapshots of each move in the game using thevisual user interface.
During the game, the diner will be served extra food items(“crosses”) besides the dish. When the diner places a food itemon the grid, the system detects its position, determines the placement of an antagonistic droplet next, and moves a droplet (as a“naught”) to the computed placement to complete its move. Notethat an existing droplet on the playground might need to moveaway from its original cell to make room for the next droplet tomove through to the targeting cell to avoid two droplets collide(Figure 21 - 4th Move). Whoever (the diner or the system) succeeds
in placing three of their marks in a horizontal, vertical, or diagonalrow (the diner or the system) is the winner. If the diner loses thegame or the game ends in a tie, the diner consumes all the fooditems in the “playground”; while if the diner wins, they get the
chance to eat the main food item at the center as a reward.

5.3

Implications of dynamic trajectories within
Dancing Delicacies dishes

Through a reflection on each of our exemplar dishes, we discuss
the implications of dynamic trajectories within designing Dancing
Delicacies dishes to provide an initial understanding of what they
could mean for future HFI design.
The instructional trajectory facilitated by the “Jelly Wagashi”
dish provided a dining experience akin to “live-cooking” (as often
the case in some fine dining restaurants where chefs cook in front of
the guests) but without chefs standing by. The chef would only needto supply selected ingredients and food items on the plate, while
the system would lead the diners to enjoy the dish according to thepre-programmed path curated by the chef. While the diner is given
some level of customizability (i.e., pick a food item), the diner’sDancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Figure 21: Flow chart of moves resulting in one possible board position when the diner makes the first move.

Figure 22: Demonstration of Tic Tac Toe in action. Our recipe: 1) Reward food: chicken nuggets, sour cream, celery; 2) Naughts:
grapefruit juice; 3) Crosses: green peas.
actions do not affect the trajectory. Hence the system ensures aconsistent quality of the dish. An instructional trajectory would
especially benefit diners who desire an optimized flavor experience
by relying on the food creator to ensure a palatable outcome. The
chef can fully demonstrate their skill and expertise, and, at the sametime, the diner will not be overwhelmed by dazzling choices.
The directional trajectory in the “Meta Platter” dish carries
out a selection and decision-based recipe, and both the diner andcreator are given joint control over the dining trajectory. The dineris provided with a collection of ingredients that they can pick asthey like. Yet, unlike Jelly Wagashi, which is guaranteed to produce
a dish based on a given recipe and a defined trajectory, here, thedecisions the diner makes may lead to a different result. In otherwords, the dish is centered around the diner’s decisions, and thechef is responsible for programming the conditions and options(i.e., the nodes of the decision tree) to produce a palatable outcome.
Such a directional trajectory implies a synergy between the dishcreator and the diner. The resulting dish design might be moresuitable for diners who are eager to explore novel taste experiences
and appreciate the opportunity to experiment with food.
With the performative trajectory, we demonstrate that foodcan enact key roles in a performance to convey stories. We notethat using food as a display and representation of information has
become increasingly popular in HFI [45-48, 62]. Existing works
often use arranged food items to display information, such as arrays of bubbles [39] or dyed/clear water in perfusable channels toshow patterns or digits [91]. Instead of simply arranging the food tovisualize information, we regard food as a computational agent thatinteracts with the inter-material properties in a performative manner. This perspective speaks to prior work, which acknowledgedthat food and performance share “the same theatrical conventions
precisely because they were both threads in the complex designof regal hospitality” [18]. Seasonal Transience offers a new way
for diners to appreciate the artistry of a performance composed bychefs and presents a new form of storytelling that can be facilitateddynamically by the dish itself. In addition to the dish’s artistic andnarrative values, the food’s movements can also be leveraged tosteer the diner’s attention or promote certain dining behaviors. For
example, if the diner becomes distracted from the dish, the food on
the plate could start to move to regain the user’s attention.
The freeform trajectory facilitated by the “Tic Tac Toe” dishused food as both input and output, in contrast with the existingfood-as-display research [62]. In particular, the droplet’s actions are
not choreographed but instead generated by an algorithm, thereby
possibly making them adaptive to the context. From the diner’s perspective, the droplets may also present an illusion of being “smart”
while retaining their authentic food properties, making dining a
fun and interactive experience while the food remains palatable.
Moreover, we envision that our system could enable the droplets
to respond to the user’s actions differently depending on the context and pre-programmed rules. We speculate that techniques likethis may help to promote desirable eating behaviors, such as using
games to retain children’s attention while dining and promoting abalanced diet by using digital play to incentivize eating vegetables[55]. We present this implementation as an exploration of using
computational food as an interactive media, and the application ofsuch systems presents a future design opportunity for “playful HFI”
[7, 61].

6

PRELIMINARY ENGAGEMENT WITH
EXPERT PRACTITIONERS

We engaged with a variety of expert practitioners by organizing an“interactive food” event and conducting expert interviews followedby the event. Our event attracted a number of expert practitioners(60 attendees) from the hospitality industry, including chefs, foodproducers, retailers, journalists, and food technology engineers, aswell as academics with an interest in interactive food. Organizing
this event allowed us to bring our design into the real-world forcommunication among disparate stakeholders, and to gain an initial
understanding of their first impressions of, and their overall attitudes and opinions toward our design. The event consisted of five

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

Jialin Deng et al.

Table 1: Participants’ details for the follow-up interviews
Participant (pseudonym)

Occupation

Years of industry
experience

Experience with food-related digital technology

Luke (48, M)

Manager of food innovation
incubator
Product developer, design
engineer (robotic food startup)
Technician/trainer in coffeeindustry
Event organizer

22

Experienced in promoting, coaching, andsupporting digital technology for food businesses
Expert in designing and developing food robotics
and automation
Uses digital technology frequently in foodpractice
No experience but knowledgeable about digitaltechnology in the food industry
No experience but has passing knowledge about
digital technology in gastronomy

Bill (30, M)
Nance (25, F)
Gabe (28, M)
Kath (31, F)

Head pastry chef, former finedining chef

15
5
5
10

sections: an “interactivity”-style demonstration (60mins) that sandwiched introductory and “interactive food” panel talks (45mins),
followed by socializing (30mins). In the demonstration session, attendees were able to engage with our Dancing Delicacies system,with the help of one researcher who provided instructions and two
researchers who assisted with programming the dish via our visualinterface to manipulate the food droplets’ movements. Attendees
were given the option to participate in follow-up interviews in amore private setting to further elaborate on their thoughts andfurther unpack their insights into their interactivity experience.
Five participants (range 25–48 years, M=32.40, S.D.=9.02) who all
had at least 5-year industry experience, volunteered for the followup interviews. The semi-structured interviews were conducted
online after the event, lasted 45-60 minutes, and were audio/video
recorded with the participants’ consent. During the interviews, wefirstly asked participants to identify and describe their preferredrole in daily food practices and their experiences with food-related
digital technology. This provided us with first-person insights into
how participants perceived themselves as a “foodie” who has an
ardent or refined interest in cooking or eating food and their experiences with using digital technology around food practices. These
responses helped us to put participants’ insights collected throughthe interviews into the context of our research focus (Table 1). Next,
based on what the participants saw and experienced at the event,
we asked several probing questions to establish a basis for open
conversations to gain insights into how experts perceive the potential values of dynamic trajectories with computational food andspeculate possible use contexts.

6.1

Findings

The interviews were transcribed and analyzed using a reflexive
thematic analysis approach [14]. We used open coding to uncover
prevalent themes among the participants’ descriptions of their experiences and speculations of use context. Our analysis resulted
in a final set of 52 codes, which we then examined for recurring
themes. Our findings encapsulate seven emerging themes and suggested a strong interest and enjoyment among participants wheninteracting with the system.

6.1.1 Theme 1: Dynamic emotion evocations in savoring the art. All
participants suggested that the Dancing Delicacies design has thepotential to offer novel dining experiences through adding a “‘wow’
factor” to “amaze the diners” (Kath). Specifically, in response to the
dynamic movements and the “aesthetic” quality of the performative
dish, Luke highlighted the concept of “emotional evocation” of foodproducts that would allow diners to “experiment with the rhythm
and contexts of the dancing based on their mood”. For example,
diners could create a “slow” or “fast dance”, with a “different cultural
type”, like a “beautiful, almost, like, an artistic experience” in a finedining context. Gabe stressed this by pointing out that “food is
not just a taste experience, it’s visual, it’s emotional, it comes as
many aspects of being a human person”. This theme extended ourreflection on the implications of a “performative” trajectory andsuggested the potential to evoke emotions when savoring the dishas an artwork.
6.1.2 Theme 2: Playfulness for family-friendly engagement. Participants mentioned that the system provided “a playful way to rethink
the experience [of eating]” (Luke). Luke added that the system could“create a fun exercise where it’s playful and the kids could program
the dessert and it brings together different flavors, as children aremuch more capable of doing gaming”. Bill also pointed out thatthe “user could control the dots [droplets] like playing Pacman,something [. . .] to make incentives or rewards around eating food”.
These statements speak to our design intention for a “freeform”
dish that uses games to retain children’s attention to promote a balanced diet. Furthermore, Bill stressed that the Dancing Delicacies
system could enable the children to engage with food and to “build
an appreciation and understanding of the food they are eating [. . .]as opposed to treat[ing] it as a chore”, so that the children and theirparents can have a good time enjoying their meals together: “that[the playfulness] strengthens the relationship with food itself” aswell as promote a “family-friendly engagement” (Bill). This theme
speaks to our reflection on the “freeform” dish which can support
future “playful HFI” design [7, 61].
6.1.3 Theme 3: Synergy from real-time concept visualization. Participants indicated that Dancing Delicacies was a “tool” that could
possibly offer a “tangible communication” (Bill) to facilitate the synergy between food creators and consumers. Bill stated that thereDancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

are “barriers around communicating concepts from the user to theprofessional language”, and the “accessibility and the consistency”
of the language are different, because “it doesn’t translate across thesame consistent language use” between diners and food creators.
In this regard, “it has [provided] a way to visibly show some processes to a diner” and can “actually illustrate specific colors, types,
and flavors of droplets” (Bill). Creators then could “dispense tiny
amounts and combine them in different ways” to show different
outcomes (Bill). This aligns with our intention of an “instructional”
dish. Additionally, the droplets could “mimic pixels that act like adisplay” that occurs in front of diners, and they could “rearrange
them [pixels] to make a different image [. . .] that gives the information straight back to the creator” (Bill). This capability for real-timevisualization could assist diners to precisely express the “nuancearound flavor” (Bill) and helps creators recommend a certain flavor
combination for the diners. This theme further confirmed that the“directional” dish could facilitate a synergy between the dish, its
creator, and the diner.

lime, or less acid, like mandarin). Similarly, Bill suggested that thesystem could automate the process of sample handling in a foodproduction pipeline, because “the motility of droplets can be used
for collecting and testing samples of beverages” (Bill), such as the“sugar content and pH level of a beverage product for quality andshelf stable lifespan control” (Bill).

6.1.4 Theme 4: Bounded taste exploration. Participants appreciatedthe “exploratory” quality of the system, particularly in a service
context where diners are given the chance to explore and try new
taste combinations. However, Bill pointed out that it might be risky
to try new things, so the exploration would need to be “still within
the bounds” of “what people are willing to, or where that decision isgonna end up”. The system could provide a purposeful and “artificial
constraint to get people to try new things”, like a bounded taste
exploration that can be “very discreet, like tiny sampling [. . .], or awinery cellar door-type experience, and sequential tasting” (Bill).
This theme appears to confirm our design of the “directional” dish.
6.1.5 Theme 5: Timely cooking process. Participants appreciatedthat Dancing Delicacies enabled new ways to assist food preparation, consumption, and production through computation. For
example, participants stated that “it would change the traditional
way of making a dish for sure” (Kath), and that Dancing Delicacies could support a “highly innovative restaurant patron and chef”(Luke) to “push the boundaries of food and what they can do tocreate new dishes” (Kath). Kath shared how she envisions new waysof preparing and cooking: “The first thing that comes to mind isthat the fats are almost always separated from most foods, like oiland vinegar. So, with this system, if you have a salad that is naked
in the middle, and then you’ve got the dressing just going around
it and mixing [to] avoid the acid being eaten into the lettuce”. Kathsuggested this timely cooking process could help creators build theperfect vinaigrette and master variations using different ingredients
to balance the flavors and emulsify the mixture.
6.1.6 Theme 6: Flavor experimentation. Nance imagined a new tasting experience through precisely dispensing droplets in a specificblending ratio: “I really enjoy different possibilities and trying different flavors and combinations”, Nance said, “for example, I canhave a certain blending ratio [. . .], like 50% sourness and 50% sweetness in a same dish or drink, you just [need to] exclude or include
some elements or flavors at some point in time on the same dish”.
Nance also pointed out that the different sweet-sour ratio reminded
her of different types of acids in different fruits, so diners could
adjust how they like to have the sour-sweet taste (more acid, like

6.1.7 Theme 7: “Fun education” for food and technology schooling.
The participants also suggested that Dancing Delicacies could support “fun education” [13] as an engaging teaching aid for food andtechnology schooling. Luke pointed out that the system could be“assembled as a tool for young people to enjoy, play, and develop”
and that its benefits go beyond “the food industry”. He felt that“having this tool be used in a tech school would be really interesting” because there are lots of “purpose-built learning centers”
for “teaching crossover STEM skills, such as food technology andprogramming” to young adults. Bill provided another perspective
regarding food education: “I could see a lot of, like, educational use
cases, like talking about combining ingredients in front of people or
showing, for example, either sequential operations or preparation
to show the effect of, like, dilution and strength of a food droplet
solution on taste, things like that”.

7

DISCUSSION: ORCHESTRATION OF
CREATOR-FOOD-DINER INTERACTIONS

Our themes suggested that the dynamic trajectories could potentially evoke emotions, promote playfulness, facilitate real-time communication, and enable exploratory eating. We see these themes
as design elements for an “orchestration” within the dining trajectories, where the dish, its creator, and the diner work together tocreate cohesive and engaging dining experiences. Such orchestration suggests a more nuanced interaction between food creator,
food and diner compared with a traditional dining experience.
Orchestration can be deployed to shape what people do andfeel in the context of a particular interaction [50]. In this regard,
prior work argued that HCI has a lot in common with theatricalconventions in terms of orchestration, including “the variables offrequency, range, and significance”, as well as “sensory immersion
and the tight coupling of kinesthetics input and visual response”[50]. This aligns with how the presentation and movements ofDancing Delicacies dishes and the procedures of food preparing
and eating were choreographed. The orchestration involves a seriesof activities (including activities of “behind the scenes” and “frontof house”) oriented toward the smooth running of the experiences[68], with creators programming (behind the scenes) and diners
reconfiguring (front of house) a dish.
These interactions open an opportunity for food creators to dynamically engage in a discourse with the diner. For example, chefs,
through their programming, could encode their concepts or a story
into a dish or a food journey, thereby offering a space of aesthetic
opportunities for diners to explore new tastes and experiences. Suchan exploratory experience could be choreographed according todiners’ personal preferences or dietary requirements. Furthermore,
the Dancing Delicacies system facilitates a tangible way of communication beyond language with diners communicating to the chef
their requirements for a meal through their interactions with thedish.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA

These qualities of computational food carry the potential to elevate the act of eating to the status of an artwork that is composedby the chef and then performed by the diner. However, the changes
in the locus of control [41] also challenge the diner-creator relationship. For example, when giving the diner more agency in the creation, the creator might see their practice of expressing themselves
as becoming limited. Our study suggested that the creators mightwant to set bounds for the diner’s agency to ensure consistency
in terms of the quality of their creations (dishes). Such challenges
could lead to power tensions between diners and creators. However,they might also lead to novel value through co-creation. We believethat our work might help identify new ways in which creators anddiners can interact and break through the traditional restaurant
separation of kitchen and dining table.

8

LIMITATIONS & FUTURE WORK

Our work utilized EWOD technology. While this technology allows
to efficiently manipulate droplets, it also limits the food materials toaqueous forms. The qualities of food droplets (i.e., surface tension
and viscosity) determine whether the intended operations can beproperly performed. Future explorations around other technologies
might offer more possibilities and hence be useful for an additional
understanding of the dynamic trajectories when designing computational food.
Also, although we have not fully explored the hardware’s capabilities in this paper, we envision a variety of future interactive
approaches to computational food via EWOD, in tandem with othermedia and input modalities. For example, EWOD technology could
be used for food testing (safety analysis), as hinted in prior work[12]. Also, the electrodes could possibly be capable of joule heating[63], which could be leveraged for precise micro-cooking on theplate. We also envisage remote interactions, as suggested by the“telematic diner” [11]. For example, a user’s action could be detectedby one system and then this information could be transmitted toanother system that reflects the change on the remote plate. We
speculate that this form of distributed computational food may foster a sense of commensality over a distance [73] and we encourage
future work in this area.
Furthermore, we have only demonstrated initial exemplar designs. We believe that many more opportunities exist for the future
development of dynamic trajectories when designing computational food via more sophisticated computational capabilities. We
also believe that further studies, along with collaborations, such as
co-design workshops with culinary practitioners, diners, and technologists, could help with the exploration of these opportunities.

9

CONCLUSION

In this paper, we presented a design-led HFI inquiry resulting in
a new system design called “Dancing Delicacies”. We provided adesign framework with four exemplar dishes to illustrate a newspace of computational food. Through a preliminary engagementwith expert practitioners from the hospitality industry, we derived
seven themes to provide a first account of understanding dynamic
dining trajectories when designing computational food. With thispaper, we wish to inspire researchers and designers to envision a
new future of human-food interaction.

Jialin Deng et al.

ACKNOWLEDGMENTS
Florian ‘Floyd’ Mueller thanks the Australian Research Council,
especially DP190102068, DP200102612 and LP210200656.

REFERENCES[1] In Japan, novelty sweets mark the seasons. The Economist (2021), Retrieved September, 2022, from: https://www.economist.com/asia/2021/08/19/in-japan-noveltysweets-mark-the-seasons.
[2] Andrew Adamatzky, 2019, A brief history of liquid computers. PhilosophicalTransactions of the Royal Society B: Biological Sciences, 374(1774): pp. 20180372.
DOI: 10.1098/rstb.2018.0372.
[3] Andrew Adamatzky, 2021. Handbook of Unconventional Computing. Handbook of
Unconventional Computing.
[4] Anand Baby Alias, Cheng-En Chiang, Hong-Yuan Huang, Kai-Ti Lin, Pei-Jhen Lu,
Yi-Wen Wang, Tzu-Hui Wu, Pei-Shin Jiang, Chien-An Chen, and Da-Jeng Yao,
2020, Extraction of Cell-free Dna from An Embryo-culture Medium Using Microscale Bio-reagents on Ewod. Scientific Reports, 10(1): pp. 9708. DOI: 10.1038/s41598020-66779-z.
[5] Mirela Alistar and Urs Gaudenz, 2017, OpenDrop: An Integrated Do-It-Yourself
Platform for Personal Use of Biochips. Bioengineering (Basel), 4(2). DOI:
10.3390/bioengineering4020045.
[6] Ferran Altarriba Bertran, Samvid Jhaveri, Rosa Lutz, Katherine Isbister, andDanielle Wilde. 2019. Making Sense of Human-Food Interaction. In Proceedingsof the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow,
Scotland Uk. Association for Computing Machinery, New York, NY, USA. Paper
678, 1-13. DOI: 10.1145/3290605.3300908.
[7] Ferran Altarriba Bertran, Danielle Wilde, Ernő Berezvay, and Katherine Isbister.
2019, Playful Human-Food Interaction Research: State of the Art and Future Directions. In Proceedings of the Annual Symposium on Computer-Human Interactionin Play, Barcelona, Spain, Association for Computing Machinery, New York, NY,
USA. 225–237. DOI: 10.1145/3311350.3347155.
[8] Judith Amores. Computational Food. (2014), Retrieved January, 2023, from: https://www.media.mit.edu/projects/computational-food/overview/.
[9] Shun An, Mingyuan Zhu, Kan Gu, Modi Jiang, Qingchen Shen, Benwei Fu,
Chengyi Song, Peng Tao, Tao Deng, and Wen Shang, 2020, Light-driven motionof water droplets with directional control on nanostructured surfaces. Nanoscale,12(7): pp. 4295-4301. DOI: 10.1039/C9NR09575F.
[10] Peter Arnold, Rohit Ashok Khot, and Florian Mueller. 2018. "You Better Eat toSurvive": Exploring Cooperative Eating in Virtual Reality Games. In Proceedingsof the Twelfth International Conference on Tangible, Embedded, and Embodied
Interaction, Stockholm, Sweden. Association for Computing Machinery, New York,NY, USA. 398–408. DOI: 10.1145/3173225.3173238.
[11] Pollie Barden, Rob Comber, David Green, Daniel Jackson, Cassim Ladha, Tom
Bartindale, Nick Bryan-Kinns, Tony Stockman, and Patrick Olivier. 2012. Telematic dinner party: designing for togetherness through play and performance. In
Proceedings of the Designing Interactive Systems Conference, Newcastle Upon Tyne,
United Kingdom. Association for Computing Machinery, New York, NY, USA.
38–47. DOI: 10.1145/2317956.2317964.
[12] S. R. Barman, I. Khan, S. Chatterjee, S. Saha, D. Choi, S. Lee, and Z. H. Lin, 2020,
Electrowetting-on-dielectric (EWOD): Current perspectives and applications inensuring food safety. J Food Drug Anal, 28(4): pp. 595-621. DOI: 10.38212/22246614.1239.
[13] Christian Bisson and John Luckner, 1996, Fun in Learning: The PedagogicalRole of Fun in Adventure Education. Journal of Experiential Education, 19(2): pp.
108-112. DOI: 10.1177/105382599601900208.
[14] Virginia Braun and Victoria Clarke, 2019, Reflecting on reflexive thematic analysis. Qualitative Research in Sport, Exercise and Health, 11(4): pp. 589-597. DOI:
10.1080/2159676X.2019.1628806.
[15] Longquan Chen and Elmar Bonaccurso, 2014, Electrowetting — From statics todynamics. Advances in Colloid and Interface Science, 210: pp. 2-12. DOI: https://doi.org/10.1016/j.cis.2013.09.007.
[16] Longquan Chen and Elmar Bonaccurso, 2014, Effects of surface wettability andliquid viscosity on the dynamic wetting of individual drops. Physical Review E,
90. DOI: 10.1103/PhysRevE.90.022401.
[17] N. J. Cira, A. Benusiglio, and M. Prakash, 2015, Vapour-mediated sensing
and motility in two-component droplets. Nature, 519(7544): pp. 446-450. DOI:
10.1038/nature14272.
[18] Denise E. Cole, Edible performance: feasting and festivity in early Tudor entertainment, in S. Banes and A. Lepecki (Eds.), The Senses in Performance (1st ed.,pp. 13). DOI: https://dx.doi.org/10.4324/9780203965924.
[19] Rob Comber, Eva Ganglbauer, Jaz Hee-jeong Choi, Jettie Hoonhout, Yvonne
Rogers, Kenton O’Hara, and Julie Maitland. 2012, Food and interaction design:designing for food in everyday life. In CHI ’12 Extended Abstracts on Human
Factors in Computing Systems, Austin, Texas, USA, Association for Computing
Machinery, New York, NY, USA. 2767–2770. DOI: 10.1145/2212776.2212716.

Dancing Delicacies: Designing Computational Food for Dynamic Dining Trajectories

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA[20] Rob Comber, Eva Ganglbauer, Jaz Choi, Jettie Hoonhout, Yvonne Rogers, Kenton
O’Hara, and Julie Maitland, 2012, Food and interaction design: Designing for foodin everyday life. Conference on Human Factors in Computing Systems - Proceedings.
DOI: 10.1145/2212776.2212716.
[21] Rob Comber, Jaz Choi, Jettie Hoonhout, and Kenton O’Hara, 2014, Designingfor human–food interaction: An introduction to the special issue on ‘food andinteraction design’. International Journal of Human-Computer Studies, 72: pp.
181-184. DOI: 10.1016/j.ijhcs.2013.09.001.
[22] Jialin Deng, Patrick Olivier, and Florian Mueller. 2021, Design of Cyber Food:
Beginning to Understand Food as Computational Artifact. In Extended Abstractsof the 2021 CHI Conference on Human Factors in Computing Systems, Yokohama,Japan, Association for Computing Machinery, New York, NY, USA. Article 293.
DOI: https://doi.org/10.1145/3411763.3451687.
[23] Jialin Deng, Yan Wang, Carlos Velasco, Ferran Altarriba Bertran, Rob Comber,Marianna Obrist, Katherine Isbister, Charles Spence, and Florian Mueller. 2021,
The Future of Human-Food Interaction. In CHI Conference on Human Factors in
Computing Systems Extended Abstracts (CHI ’21), Yokohama, Japan, ACM, New
York, NY, USA. DOI: 10.1145/3411763.3441312.
[24] Jialin Deng, Ferran Altarriba Bertran, Lining Yao, Marianna Obrist, Koya Narumi,
Humphrey Yang, Mako Miyatake, and Florian Mueller. 2022, Mapping FoodHCI
Futures. In CHI Conference on Human Factors in Computing Systems, New Orleans,
LA, USA, Association for Computing Machinery, New York, NY, USA. 1–5. DOI:
https://doi.org/10.1145/3491101.3516401.
[25] Jialin Deng, Patrick Olivier, Josh Andres, Kirsten Ellis, Ryan Wee, and Florian
Mueller. 2022, Logic Bonbon: Exploring Food as Computational Artifact. In CHI
Conference on Human Factors in Computing Systems (CHI’22), New Orleans, LA,
USA, ACM, New York, NY, USA. 21 pages. DOI: https://doi.org/10.1145/3491102.
3501926.
[26] Jialin Deng, Ferran Altarriba Bertran, Marianna Obrist, Yan Wang, Florian ‘Floyd’
Mueller, and Carlos Velasco, 2023, Sketching the future of human-food interaction:
Emerging directions for future practice. International Journal of Gastronomy andFood Science, 31: pp. 100629. DOI: https://doi.org/10.1016/j.ijgfs.2022.100629.
[27] Jialin Deng, Patrick Olivier, and Florian ‘Floyd’ Mueller, 2023, Exploring tasty
fluidics for designing food as computational artifact. International Journal of
Gastronomy and Food Science, 31: pp. 100630. DOI: https://doi.org/10.1016/j.ijgfs.
2022.100630.
[28] Anthony Dunne and Fiona Raby, 2013. Speculative everything: design, fiction, andsocial dreaming. MIT press.
[29] Tom H. Fisher, 2004, What We Touch, Touches Us: Materials, Affects, and Affordances. Design Issues, 20(4): pp. 20-31. DOI: 10.1162/0747936042312066.
[30] Florence. A guide to the Japanese sweets for every season. Go! Go! Nihon (2020), Retrieved September, 2022, from: https://gogonihon.com/en/blog/japanese-sweetsfor-every-season/.
[31] William Gaver, What should we expect from research through design?, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(pp. 937–946). Association for Computing Machinery. DOI: https://dx.doi.org/10.
1145/2207676.2208538.
[32] William W. Gaver and Heather Martin. 2000, Alternatives: exploring information
appliances through conceptual design proposals. In Proceedings of the SIGCHI
conference on Human Factors in Computing Systems (CHI ’00), Association for
Computing Machinery, New York, NY, USA. 209–216. DOI: https://doi.org/10.
1145/332040.332433.
[33] Hongyao Geng, Jian Feng, Lisa Marie Stabryla, and Sung Kwon Cho, 2017, Dielectrowetting manipulation for digital microfluidics: creating, transporting,splitting, and merging of droplets. Lab on a Chip, 17(6): pp. 1060-1068. DOI:
10.1039/C7LC00006E.
[34] Andrea Grimes and Richard Harper. 2008, Celebratory technology: new directions
for food research in HCI. In Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems (CHI ’08), Florence, Italy, Association for Computing
Machinery, New York, NY, USA. 467–476. DOI: 10.1145/1357054.1357130.
[35] Barry Callbaut Group. Launch event of world’s first 3D Printing Studio to craft
unseen chocolate experiences. Youtube (2020), from: https://www.youtube.com/
watch?v$=$lpjouddaGuY.
[36] Chonglei Hao, Yahua Liu, Xuemei Chen, Yuncheng He, Qiusheng Li, K. Y. Li,
and Zuankai Wang, 2014, Electrowetting on liquid-infused film (EWOLF): Complete reversibility and controlled droplet oscillation suppression for fast opticalimaging. Scientific Reports, 4(1): pp. 6846. DOI: 10.1038/srep06846.
[37] Florian Heller. 2021, Muffidgets: Detecting and Identifying Edible Pastry Tangibles on Capacitive Touchscreens. In Proceedings of the Fifteenth International
Conference on Tangible, Embedded, and Embodied Interaction (TEI’21), Salzburg,
Austria, Association for Computing Machinery, New York, NY, USA. Article 44.
DOI: 10.1145/3430524.3442449.
[38] Do Jin Im, Jihoon Noh, Dustin Moon, and In Seok Kang, 2011, Electrophoresisof a Charged Droplet in a Dielectric Liquid for Droplet Actuation. Analytical
Chemistry, 83(13): pp. 5168-5174. DOI: 10.1021/ac200248x.
[39] Ayaka Ishii and Itiro Siio. 2019, BubBowl: Display Vessel Using Electrolysis Bubbles in Drinkable Beverages. In Proceedings of the 32nd Annual ACM Symposium

on User Interface Software and Technology (UIST ’19), Association for Computing
Machinery, New York, NY, USA. 619–623. DOI: https://doi.org/10.1145/3332165.
3347923.
[40] Vandana Jain, Amit Hole, Raghavendra Deshmukh, and Rajendra Patrikar, 2017,
Dynamic capacitive sensing of droplet parameters in a low-cost open EWOD
system. Sensors and Actuators A: Physical, 263: pp. 224-233. DOI: https://doi.org/
10.1016/j.sna.2017.06.014.
[41] Jinkyu Jang, Hyeonsik Shin, Hajung Aum, Minji Kim, and Jinwoo Kim, 2016,
Application of experiential locus of control to understand users’ judgments
toward useful experience. Computers in Human Behavior, 54: pp. 326-340. DOI:
https://doi.org/10.1016/j.chb.2015.08.010.
[42] Viirj Kan, Judith Amores, Chikara Inamura, Yujie Hong, Dhruv Jain, and Hiroshi
Ishii. 2014, Food as Prototype for Radical Atoms. In In submission to MAS 834:
Tangible Interfaces final project report.
[43] Georgios Katsikis, James S. Cybulski, and Manu Prakash, 2015, Synchronous
universal droplet logic and control. Nature Physics, 11(7): pp. 588-596. DOI:
10.1038/nphys3341.
[44] Rohit Khot, Florian Mueller, and Damon Young, 2019, Human-Food Interaction.
Foundations and Trends®in Human-Computer Interaction, 12: pp. 238-415. DOI:
10.1561/1100000074.
[45] Rohit Ashok Khot, Jeewon Lee, Larissa Hjorth, and Florian ’Floyd’ Mueller. 2015,
TastyBeats: Celebrating Heart Rate Data with a Drinkable Spectacle. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied
Interaction (TEI ’15), Stanford, California, USA, Association for Computing Machinery, New York, NY, USA. 229–232. DOI: 10.1145/2677199.2680545.
[46] Rohit Ashok Khot, Ryan Pennings, and Florian ’Floyd’ Mueller. 2015, EdiPulse:
Supporting Physical Activity with Chocolate Printed Messages. In Proceedings ofthe 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15), Seoul, Republic of Korea, Association for Computing
Machinery, New York, NY, USA. 1391–1396. DOI: 10.1145/2702613.2732761.
[47] Rohit Ashok Khot, Deepti Aggarwal, Ryan Pennings, Larissa Hjorth, and Florian
Mueller. 2017. Edipulse: investigating a playful approach to self-monitoring
through 3D printed chocolate treats. In Proceedings of the 2017 CHI Conference on
Human Factors in Computing Systems, 6593-6607. DOI: 10.1145/3025453.3025980.
[48] Rohit Ashok Khot, Larissa Hjorth, and Florian Mueller, 2020, Shelfie: A Framework for Designing Material Representations of Physical Activity Data. ACM
Transactions on Computer-Human Interaction: pp. 1-52. DOI: 10.1145/3379539.
[49] Tom Krupenkin and J. Ashley Taylor, 2011, Reverse electrowetting as a new
approach to high-power energy harvesting. Nature Communications, 2(1): pp. 448.
DOI: 10.1038/ncomms1454.
[50] Brenda Laurel, 1993. Computers as Theatre (2nd ed). Addison-Wesley Longman
Publishing Co., Inc., USA.
[51] Christian Østergaard Laursen, Søren Pedersen, Timothy Merritt, and Ole Caprani,
2016, Springer International Publishing, Cham, Robot-Supported Food Experiences. International Workshop in Cultural Robotics, vol 9549 pp. 107-130. DOI:
10.1007/978-3-319-42945-8_10.
[52] Andreas Lendlein and Oliver E. C. Gould, 2019, Reprogrammable recovery andactuation behaviour of shape-memory polymers. Nature Reviews Materials, 4(2):pp. 116-133. DOI: 10.1038/s41578-018-0078-8.
[53] An Li, Huizeng Li, Zheng Li, Zhipeng Zhao, Kaixuan Li, Mingzhu Li, and YanlinSong, 2020, Programmable droplet manipulation by a magnetic-actuated robot.
Science Advances, 6(7): pp. eaay5808. DOI: doi:10.1126/sciadv.aay5808.
[54] Yiyan Li, Hongzhong Li, and R. Jacob Baker, 2015, A Low-Cost and HighResolution Droplet Position Detector for an Intelligent Electrowetting on Dielectric Device. Journal of Laboratory Automation, 20(6): pp. 663-669. DOI:
10.1177/2211068214566940.
[55] Weiwei Ma, Bo Liu, and Zhao Liu. 2021, Influence of a Video Game on Children’s
Attention to Food: Should Games Be Served with a Character During Mealtime?
In HCI in Games: Serious and Immersive Games: Third International Conference,
HCI-Games 2021, Held as Part of the 23rd HCI International Conference, HCII 2021,
Virtual Event, July 24–29, 2021, Proceedings, Part II, Springer-Verlag. 42–50. DOI:
https://doi.org/10.1007/978-3-030-77414-1_4.
[56] Robert Malinowski, Ivan P. Parkin, and Giorgio Volpe, 2020, Advances towards
programmable droplet transport on solid surfaces and its applications. Chemical
Society Reviews, 49(22): pp. 7879-7892. DOI: 10.1039/D0CS00268B.
[57] Ben Matthews and Stephan Wensveen, Prototypes and prototyping in design research, in P.A. Rodgers and J. Yee (Eds.), Routledge Companion to Design Research(pp. 262-276). Routledge. DOI: https://dx.doi.org/10.4324/9781315758466-25.
[58] Yash Dhanpal Mehta, Rohit Ashok Khot, Rakesh Patibanda, and Florian Mueller.
2018. Arm-A-Dine: Towards Understanding the Design of Playful Embodied
Eating Experiences. In Proceedings of the 2018 Annual Symposium on ComputerHuman Interaction in Play, Melbourne, VIC, Australia. Association for Computing
Machinery, New York, NY, USA. 299–313. DOI: 10.1145/3242671.3242710.
[59] Moran Mizrahi, Amos Golan, Ariel Bezaleli Mizrahi, Rotem Gruber, Alexander
Zoonder Lachnise, and Amit Zoran. 2016, Digital Gastronomy: Methods & Recipes
for Hybrid Cooking. In In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ’16), Tokyo, Japan, Association for Computing
Machinery, New York, NY, USA. 541–552. DOI: 10.1145/2984511.2984528.

DIS ’23, July 10–14, 2023, Pittsburgh, PA, USA[60] Florian Mueller, Richard Byrne, Josh Andres, and Rakesh Patibanda, 2018. Experiencing the Body as Play. ACM. 1-13.
[61] Florian Mueller, Yan Wang, Zhuying Li, Tuomas Kari, Peter Arnold, Yash Mehta,
Jonathan Marquez, and Rohit Khot. 2020. Towards Experiencing Eating as Play.
In Proceedings of the ACM Tangible, Embedded, & Embodied Interaction Conference(TEI’20), February 9–12, Sydney, NSW, Australia. ACM, New York, NY, USA. 239253. DOI: 10.1145/3374920.3374930.
[62] Florian Mueller, Khot Rohit Ashok, Dwyer Tim, Goodwin Sarah, Marriott Kim,
Jialin Deng, Phan Han D, Jionghao Lin, Kun-Ting Chen, and Yan Wang. 2021,
Data as Delight: Eating data. In Proceedings of the 2021 CHI Conference on Human
Factors in Computing Systems (CHI’21), Yokohama, Japan, ACM, New York, NY,
USA. DOI: https://doi.org/10.1145/3411764.3445218.
[63] Krishnadas Narayanan Nampoothiri, Mahadevan Subramanya Seshasayee, Vinod
Srinivasan, M. S. Bobji, and Prosenjit Sen, 2018, Direct heating of aqueous droplets
using high frequency voltage signals on an EWOD platform. Sensors and Actuators
B: Chemical, 273: pp. 862-872. DOI: https://doi.org/10.1016/j.snb.2018.06.091.
[64] Emad Naseri, Anthony Van Beek, and Ali Ahmadi, 2022, Application of electrowetting on dielectric (EWOD) in drug release control and release-on-demand.
Colloids and Surfaces A: Physicochemical and Engineering Aspects, 654: pp. 130131.
DOI: https://doi.org/10.1016/j.colsurfa.2022.130131.
[65] A. Niijima and T. Ogawa. 2016, A proposal of virtual food texture by electricmuscle stimulation. In 2016 IEEE International Conference on Multimedia & Expo
Workshops (ICMEW), Seattle, WA, 1-6. DOI: 10.1109/ICMEW.2016.7574698.
[66] Yumi Nishihara and Yasuaki Kakehi. 2021, magashi: Fabrication of ShapeChanging Edible Structures by Extrusion-Based Printing and Baking. In Creativity and Cognition (C&C ’21), Virtual Event, Italy, Association for Computing
Machinery, New York, NY, USA. Article 44. DOI: 10.1145/3450741.3465388.
[67] Xianming Qin, Xueyong Wei, Lei Li, Hairong Wang, Zhuangde Jiang, and Dong
Sun, 2021, Acoustic valves in microfluidic channels for droplet manipulation. Lab
on a Chip, 21(16): pp. 3165-3173. DOI: 10.1039/D1LC00261A.
[68] Stuart Reeves, Steve Benford, Claire O’Malley, and Mike Fraser. 2005, Designing
the spectator experience. In Proceedings of the SIGCHI Conference on Human
Factors in Computing Systems (CHI ’05), Portland, Oregon, USA, Association for
Computing Machinery, New York, NY, USA. 741–750. DOI: https://doi.org/10.
1145/1054972.1055074.
[69] Sabiha Samad, Fahmida Ahmed, Samsun Naher, Muhammad Ashad Kabir, Anik
Das, Sumaiya Amin, and Sheikh Mohammed Shariful Islam, 2022, Smartphone
apps for tracking food consumption and recommendations: Evaluating artificial
intelligence-based functionalities, features and quality of current apps. Intelligent
Systems with Applications, 15: pp. 200103. DOI: https://doi.org/10.1016/j.iswa.2022.
200103.
[70] Robert Simonson. A Mood-Ring Ingredient Makes Cocktails Change Color. The
New York Times (2016), Retrieved March, 2022, from: https://www.nytimes.com/
2016/07/06/dining/blue-cocktails-blure-butterfly-pea-flower.html.
[71] Tibbits Skylar, 2021. Things Fall Together: A Guide to the New Materials Revolution.
Princeton University Press. 1.
[72] Huijie Song, Zizheng Fang, Binjie Jin, Pengju Pan, Qian Zhao, and Tao Xie, 2019,
Synergetic Chemical and Physical Programming for Reversible Shape Memory
Effect in a Dynamic Covalent Network with Two Crystalline Phases. ACS Macro
Letters, 8(6): pp. 682-686. DOI: 10.1021/acsmacrolett.9b00291.
[73] Charles Spence, Maurizio Mancini, and Gijs Huisman, 2019, Digital Commensality: Eating and Drinking in the Company of Technology. Frontiers in Psychology,10(2252). DOI: 10.3389/fpsyg.2019.02252.
[74] Charles Spence, Jozef Youssef, and Gustav Kuhn, 2020, Magic on the Menu: Where
Are All the Magical Food and Beverage Experiences? Foods, 9(3): pp. 257. DOI:
10.3390/foods9030257[75] Susan Stepney, 2008, The neglected pillar of material computation. Physica D:
Nonlinear Phenomena, 237(9): pp. 1157-1164. DOI: https://doi.org/10.1016/j.physd.
2008.01.028.
[76] Jie Sun, Zhuo Peng, Weibiao Zhou, Jerry Y. H. Fuh, Geok Soon Hong, and Annette
Chiu, 2015, A Review on 3D Printing for Customized Food Fabrication. Procedia
Manufacturing, 1: pp. 308-319. DOI: 10.1016/j.promfg.2015.09.057.
[77] Hervé This, 2009, Molecular Gastronomy, a Scientific Look at Cooking. Accountsof Chemical Research, 42(5): pp. 575-583. DOI: 10.1021/ar8002078.
[78] Udayan Umapathi, Droplet IO: Programmable Droplets for Human-MaterialInteraction, in Media Arts and Sciences, School of Architecture and Planning.
2017, Massachusetts Institute of Technology. p. 79.
[79] Udayan Umapathi, Samantha Chin, Patrick Shin, Dimitris Koutentakis, andHiroshi Ishii, 2018, Scaling Electrowetting with Printed Circuit Boards for

Jialin Deng et al.

Large Area Droplet Manipulation. MRS Advances, 3(26): pp. 1475-1483. DOI:
https://doi.org/10.1557/adv.2018.331.
[80] Udayan Umapathi, Patrick Shin, Ken Nakagaki, Daniel Leithinger, and Hiroshi
Ishii. 2018, Programmable Droplets for Interaction. In Extended Abstracts ofthe 2018 CHI Conference on Human Factors in Computing Systems (CHI EA ’18),
Montreal QC, Canada, Association for Computing Machinery, New York, NY,
USA. Paper VS15. DOI: https://doi.org/10.1145/3170427.3186607.
[81] Anna Vallgårda and Johan Redström. 2007, Computational composites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI’07), San Jose, California, USA, Association for Computing Machinery, New York,NY, USA. 513–522. DOI: 10.1145/1240624.1240706.
[82] Anna Vallgårda and Tomas Sokoler. 2009, A material focus: exploring propertiesof computational composites. In CHI ’09 Extended Abstracts on Human Factors in
Computing Systems (CHI EA ’09), Boston, MA, USA, Association for Computing
Machinery, New York, NY, USA. 4147–4152. DOI: 10.1145/1520340.1520631.
[83] Chi Thanh Vi, Asier Marzo, Damien Ablart, Gianluca Memoli, Sriram Subramanian, Bruce Drinkwater, and Marianna Obrist. 2017. TastyFloats: A Contactless
Food Delivery System. In Proceedings of the 2017 ACM International Conference on
Interactive Surfaces and Spaces, Brighton, United Kingdom. Association for Computing Machinery, New York, NY, USA. 161–170. DOI: 10.1145/3132272.3134123.
[84] Chi Thanh Vi, Asier Marzo, Gianluca Memoli, Emanuela Maggioni, Damien
Ablart, Martin Yeomans, and Marianna Obrist, 2020, LeviSense: A platform forthe multisensory integration in levitating food and insights into its effect on
flavour perception. International Journal of Human-Computer Studies, 139: pp.
102428. DOI: https://doi.org/10.1016/j.ijhcs.2020.102428.
[85] Ron Wakkary, William Odom, Sabrina Hauser, Garnet Hertz, and Henry Lin.
2015. Material speculation: actual artifacts for critical inquiry. In Proceedings of
the Aarhus Conference on Critical Alternatives, DOI: 10.7146/AAHCC.V1I1.21299.
[86] Wen Wang, Lining Yao, Teng Zhang, Chin-Yi Cheng, Daniel Levine, and Hiroshi Ishii. 2017. Transformative appetite: shape-changing food transforms
from 2D to 3D by water interaction through cooking. In Proceedings of the the2017 CHI Conference on Human Factors in Computing Systems, 6123-6132. DOI:
10.1145/3025453.3026019.
[87] Yan Wang, Zhuying Li, Robert Jarvis, Rohit Ashok Khot, and Florian Mueller.
2018, The Singing Carrot: Designing Playful Experiences with Food Sounds. In
Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play
Companion Extended Abstracts (CHI PLAY ’18 Extended Abstracts), Melbourne, VIC,
Australia, Association for Computing Machinery, New York, NY, USA. 669–676.
DOI: 10.1145/3270316.3271512.
[88] Yan Wang, Zhuying Li, Robert S. Jarvis, Joseph La Delfa, Rohit Ashok Khot,
and Florian Mueller. 2020, WeScream! Toward Understanding the Design ofPlayful Social Gustosonic Experiences with Ice Cream (DIS ’20). In Proceedingsof the 2020 ACM Designing Interactive Systems Conference, Eindhoven, Netherlands, Association for Computing Machinery, New York, NY, USA. 951–963. DOI:
10.1145/3357236.3395456.
[89] M. Wiberg, 2018. The Materiality of Interaction: Notes on the Materials of Interaction Design. MIT Press.
[90] Mikael Wiberg and Erik Stolterman, 2021, Time and Temporality in HCI Research.
Interacting with Computers, 33(3): pp. 250-270. DOI: 10.1093/iwc/iwab025.
[91] Humphrey Yang, Danli Luo, Kuanren Qian, and Lining Yao, Freeform Fabricationof Fluidic Edible Materials, in Proceedings of the 2021 CHI Conference on Human
Factors in Computing Systems (pp. Article 620). Association for Computing
Machinery, New York, NY, USA. DOI: https://dx.doi.org/10.1145/3411764.3445097.
[92] Lining Yao, Jifei Ou, Chin-Yi Cheng, Helene Steiner, Wen Wang, Guanyun Wang,and Hiroshi Ishii. 2015. bioLogic: Natto Cells as Nanoactuators for Shape Changing Interfaces. In Proceedings of the 33rd Annual ACM Conference on Human
Factors in Computing Systems, Seoul, Republic of Korea. Association for Computing Machinery. 1–10. DOI: 10.1145/2702123.2702611.
[93] Leslie Yeo and Hsueh-Chia Chang, Electrowetting, in D. Li (Ed.), Encyclopedia ofMicrofluidics and Nanofluidics (pp. 600-606). Springer US. DOI: https://dx.doi.
org/10.1007/978-0-387-48998-8_470.
[94] John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007, Research throughdesign as a method for interaction design research in HCI. In Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems (CHI ’07), San Jose,
California, USA, Association for Computing Machinery, New York, NY, USA.
493–502. DOI: 10.1145/1240624.1240704.
[95] John Zimmerman and Jodi Forlizzi, Research Through Design in HCI, in J.S.
Olson and W.A. Kellogg (Eds.), Ways of Knowing in HCI (pp. 167-189). Springer
New York. DOI: https://dx.doi.org/10.1007/978-1-4939-0378-8_8.