Visual Methods for the Design of Shape-Changing Interfaces

Miriam Sturdee, Aluna Everitt, Joseph Lindley, Paul Coulton, Jason Alexander

► To cite this version:

Miriam Sturdee, Aluna Everitt, Joseph Lindley, Paul Coulton, Jason Alexander. Visual Methods for the Design of Shape-Changing Interfaces. 17th IFIP Conference on Human-Computer Interaction (INTERACT), Sep 2019, Paphos, Cyprus. pp.337-358, 10.1007/978-3-030-29387-1_19. hal-02553913

HAL Id: hal-02553913

https://inria.hal.science/hal-02553913

Submitted on 24 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Visual Methods for the Design of
Shape-Changing Interfaces

Miriam Sturdee[0000-0002-8417-191X], Aluna Everitt[0000-0002-0974-2559], Joseph Lindley[0000-0002-5527-3028], Paul Coulton[0000-0001-5938-4393], and Jason Alexander[0000-0001-6093-6116]

Lancaster University, Bailrigg, Lancaster LA1 4YW, UK (m.sturdee, a.everitt, j.lindley, p.coulton, j.alexander)@lancaster.ac.uk
www.lancaster.ac.uk

**Abstract.** Shape-changing interfaces use physical change in shape as input and/or output. As the field matures, it will move from technology-driven design toward more formal processes. However, this is challenging: end-users are not aware of the capabilities of shape-change, devices are difficult to demonstrate, and presenting single systems can 'trap' user-thinking into particular forms. It is crucial to ensure this technology is developed with requirements in mind to ensure successful end-user experiences. To address this challenge, we developed and tested (n=50) an approach that combines low-fidelity white-box prototypes and high-fidelity video footage with end-user diagram and scenario sketching to design context dependent devices. We analysed the outputs of our test process and identified themes in device design requirements, and from this constructed a shape-change stack model to support practitioners in developing, classifying, and synthesising end-user requirements for this novel technology.

**Keywords:** Shape-Changing Interfaces · Sketching · Visual Methods

## 1 Introduction

Shape-changing interfaces are complex, tangible interactive objects, surfaces and spaces allowing rich computational experiences and physicalization of information. Examples include mobile phones that bend at the corners to alert you to a call or a text [39], table top surfaces that raise up to pass you your tablet [9], and the organic, liquid metal art installations that move in ever increasing complexity [24]. Shape-changing interfaces allow dynamic shape as physical input and output, as well as supporting other sensory interaction, and herald the next stage in computing hardware. At present the field is largely technology driven, with end-user applications emerging from the affordances of the available platform. Despite the diverse range of prototypes researchers have developed [55], there are no formal methods, guidelines, or tool-kits specifically developed for shape-changing interface development – with the exception of rapid prototyping, often using modular devices [20].
For existing systems, User-Centered Design ensures that the tasks, needs, and context of end-users drive and reflect upon the development of a new system, but we cannot presently apply this to shape-changing interfaces. This is because current shape-changing interface design is not targeted at solving a particular problem, or trying to design specific hardware or interaction – we are purely striving for innovation. In addition to this, possible end users are not aware of shape-change as a technology, what it can do, and the range of available hardware is difficult to demonstrate due to issues of location, portability or safety. In the cases where users are given the opportunity to interact with a shape-changing interface prototype, they may also become “trapped” into thinking about shape-change as being of one particular form (e.g. actuated pins [6]). Much of the work on shape-change appears to pick from different areas of existing design processes, but does not seek to employ them as a specific methodology during the research process: for example, building hardware is often the first step in exploring shape-changing interfaces, and is then followed by a short usability study for whichever application best suits the platform [54] [58]; or studies might focus on user-ideation or co-design for non-specific products [5] [56]. The reason behind this may be that, traditional, sequential processes (e.g. planning, user research, user evaluation, information architecture) may not fit exactly with emergent hardware which does not already have a predefined role.

At this early stage, we hope to utilise readily available methods to provide a baseline of requirements for shape-changing interfaces. By focusing on a practical start-point – requirements generation – we believe we can begin to adapt and build formal design process for this exciting technology from the ground up. Following the generation of requirements, we can model these to form an overview of the field and its possibilities. This paper therefore contributes: 1) A practical, readily available approach for requirements generation for shape-changing interfaces; 2) A 50 participant study that demonstrates the validity of this approach to generate requirements for this novel technology; (3) A thematic analysis of the generated user requirements; (4) A shape-change stack model to support practitioners in design requirements-gathering activities for shape-changing interfaces, intended to provide a cohesive resource for those building and testing shape-changing interfaces with the view to their eventual adoption.

## 2 Related Work

Generating requirements for devices that do not yet exist is an exciting challenge. To address this, we suggest utilising accessible techniques in order to inform and engage potential end-users about shape-changing interfaces. Understanding current applications and approaches to shape-change – alongside these already validated techniques – can assist in beginning to design and develop formal processes for these novel devices.
## 2.1 Shape-Changing Interfaces

Shape-changing interfaces are an exciting subset of tangible computing hardware, with the potential to not only use 3D form in input/output, but to also allow for this form to be manipulated by the user, and sometimes, self actuated. Examples include (but are not limited to): interactive tabletop surfaces such as *InFORM* [9] or the elastic-pin hybrid *Tablehop* [49]; rapid prototyping devices like *ShapeClip* [20]; mobile phones like *WhammyPhone* which explores bendable audio interaction [14] or *Reflex* which uses the bend action to enhance reading on small displays [54]; furniture such as the unexpectedly actuating *Co-Motion* bench [17]; or even public installations like *Aegis Hyposurface* [16] and *Protrude/Flow* [24]. Although a large number of works focus on hardware development, there also exist examples which periodically review, critique and ponder the state of the field of shape-change, notably Rasmussen et al. [44], Coelho & Zigelbaum [3], and recently, Sturdee & Alexander [55] but these offer general overviews of interactive capabilities or material properties rather than advice on building these novel devices (although the latter suggests material properties may be a helpful starting point for a design framework). Additionally, research touches upon particular aspects of design for these devices such as emotional or anthropometric content [27], vocabularies for design [45] or constructing form language for shape-change [61], but there still exists a gap in the literature for a consolidated framework or process for the building and development of shape-changing interfaces.

## 2.2 Requirements for Shape-Change

Requirements are the things a system should have in order to function and fulfil the needs of the user [26]. They can be gathered in a systematic (as with software requirements engineering) [52] or informal manner, with a preference for the former. However requirements engineering has not yet been directly applied to shape-change, and there are no existing parameters. Stakeholders (in this case potential users) have no existing schema for shape-change, so first the concept and structures must be communicated and the solution space expanded. For shape-change, the easiest way is to demonstrate and allow interaction with existing hardware, but this is not practical given constraints such as geographical location and accessing multiple devices from different research labs. The second challenge is how to capture the stakeholder responses as you cannot simply interview the user about their experiences with a product that does not exist, and that they have not used. To overcome these barriers, we might explore the possibility of using creative methods [38] to elicit early stage requirements in three ways: by employing an efficient method of communication [62] to describe the state-of-the-art of shape-change; by creating opportunities for interaction [41]; and by using an accessible method of information production [2].
## 2.3 Accessible Techniques for Design Requirements Generation

The techniques described below enable us to address the challenge of generating design requirements for shape-change.

**Video** High-quality video is often produced alongside published work in order provide quick explanations of a hardware or system concept, without reading the accompanying text. It can also be used to inform, communicate or explore concepts [62]. To a large percentage of users, receiving information in this way is a normal, accessible part of smart (and other) device interaction (such as *YouTube* or *Vimeo*) [10]. For actuated prototypes, a realistic rendering style was found to be the optimal way of communicating a concept to users in a study of shape-changing phones [40], whereas Gong et al. [15] suggest that high quality videos depicting novel hardware allow users to “suspend their disbelief” and make judgements about how useful a prospective technology might be. Videos have also been used within studies in combination with low-fidelity prototypes to generate high-level comments [34] which suggests that combining this media with other techniques may yield useful results. In context, video also enables us to present work that we do not have access to, due to geographical, or other constraints, and is an apt method to communicate high-level concepts.

**Low Fidelity Prototypes** Low-fidelity prototypes are quick mock-ups of designs or devices allowing concept testing without committing to an expensive or lengthy build, making them ideal in the requirements-gathering stage of the design process [48]. In HCI, concepts such as *paper-prototyping* [51] and *rapid-prototyping* [20] are often used, examined and critiqued for their role in research. For shape-change, the difficulty in creating low-fidelity prototypes is mirrored by the range of technically complex hardware and interactive capabilities of the high-fidelity, working prototypes. By thinking about the *materiality* of shape-change however, we can emphasise its tangible nature in a simple, easy-to-build manner. Schmid et al. suggests and tests a form-first approach using glass objects to generate ideas for tangible interfaces [50]. The reasoning behind using low-fidelity, *white box*, prototypes to explore shape-change are twofold: 1) Existing prototypes are often bulky, heavy, expensive and situated in laboratories across the globe; 2) White box prototypes allow for the presentation of matter in a consistent way (i.e. all the same size, colour) so that participants are unbiased by incidental details. Examples include: Kwak's *Repertory Grid Study* [27] where a variety of actuated white box prototypes were created to explore the expressive and emotional qualities in shape-change; Petrelli's work on tangible interfaces which looked at the psychological affect inherent with concepts of shape and haptic interaction [41]; and, Winther et al. who generated white box prototypes following an exploration of form language for shape-changing interfaces [61]. The recent shift in HCI toward the importance of *materials* [11] also suggests there are benefits in considering our most simple, tactile interactions, (e.g. Atkinson et al.'s consideration of natural, gestural interaction with soft materials [1]). White
box prototypes are helpful, as humans may require material *anchors* to be, by
their nature, “sketchy” in order to facilitate cognitive processes [35].

**Sketching and Storyboards** Sketches are often seen as low-fidelity – they are rough ideas that welcome opinion and modification [7], and also explain concepts that are hard to suggest with words [12]. Sketching has long been part of the user-centred design process, and Buxton’s book *Sketching User Experiences* has actively encouraged and enhanced researcher engagement with this format [2]. Sketches are also cheap to produce, and are an inclusive way of generating output as they require only access to a pen and paper – and have additional cognitive benefits [13]. Sketching also has an established place in the design of user interfaces (UI), giving rise to computer-based UI design programs which either utilise, or appear to be sketches and storyboards [18] [28] and can be annotated or embedded with metadata [19]. Storyboards used in the design process can also lead to more effective design [60] or help communicate research findings [22]. There is already a precedence for using storyboards to generate requirements, whether via sketched, computationally enhanced outputs [19] or the traditional, hand drawn versions [57], and participatory sketching has also been shown to help generate requirements for real-world interaction (e.g. elevators) [59]. Storyboards and comics are also already used in HCI within areas such as software engineering [60] and cyber-security [30] [63], and are accessible, quick-to-produce medium for proposing future scenarios. Finally, sketching in direct application to shape-change has already been employed in the user ideation process [56], as a way of exploring the design of interactions for shape-changing devices [45] and even to examine a futuristic material [23].

**3 Method**

To generate requirements for shape-changing interfaces, we asked participants to
experience the materiality of shape-change [55] with white-box prototypes (Fig. 1), and then sketch ideas, diagrams and scenarios. We took inspiration from the
described techniques, and Read et al.’s approach to exploring organic user interfaces (comics and material prototyping used in combination to communicate
ideas about materiality and change [46]) – for shape-change we can blend parts
of the analytic and design stages where feasible. Shape-change is diverse in its
materiality and potential interaction range, so to attempt to represent and explain this technology in a simple, single step would be prohibitive. We used a
combination of videos to inform and educate, low-fidelity, white-box prototypes
to enable exploration and basic interaction, and sketched output in the form of
diagrams and storyboards to both assist in explanation and later, interpretation
(the diagrams play the role of annotation or metadata for the storyboards [18]).
The desired outcome of this process is to “reduce the distance” between the
researcher and the end user [37] and create a meaningful collaboration.
**3.1 Study Overview**

The study was a five-stage process lasting between 40 minutes to 1 hour, including explanation, questions and feedback: 1) Introduction to shape-change using video material from existing research; 2) Exploration and interaction with white box prototypes; 3) Idea generation; 4) Idea elaboration and diagram creation; 5) Storyboarding and scenario generation. The participant output was collected for coding and analysis.

**3.2 Participants**

Fifty participants (24 male/26 female) were recruited using social media, email, or from volunteering after observing the study set-up directly in shared social/study spaces. For example, 8 participants responded to a pre-organised workshop call hosted internally within the university, another 12 participated after approaching the study team voluntarily in a shared postgraduate student study and social space, and 8 were invited via email and snowball sampling to participate whilst the study was set up in a general-use study-hub meeting room on the university campus. Participants with diverse social and professional backgrounds took part (of the 50, 22 were not involved in academic research or study, and 28 were either students or university staff). Those participants who were not affiliated with the university were recruited via email and word of mouth from existing social and professional relationships of the study team, and encompassed backgrounds as diverse as call-centre worker, marketing manager and retiree. The age range of participants was 21-69, (mean 49).

**3.3 Video Material**

Participants were shown 7 videos relating to existing research, chosen for quality, specific actuation type and related to the material properties of white box prototypes. The videos served to inform those taking part about the state-of-the-art in shape-change research, and introduce the concepts of materiality in prototyping. The chosen works were: *Physical Telepresence* [29] (actuated interface); *Protrude, Flow* [24] (liquid interface); *Lightcloth* [21] (paper/cloth interface); *Re-Flex* [54] (bendable interface); *Paddle* [43] (foldable interface); *Claytric Surface* [36] (malleable interface); *Obake* [4] (elastic/inflatable interface).

**3.4 White Box Prototypes**

We created 7 white box prototypes reflecting the materiality of a range of existing shape-changing interfaces representative of current functional prototypes within the field [55], one of which also demonstrated the ability of these interfaces to change state via jamming [8] – see Fig. 1. for categories and images. By utilising white-box prototypes spanning a range of materials, we can communicate the intended interaction of shape-changing interfaces in a simple, portable manner.
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** White-box prototypes based on shape-change categories from [55], with examples of how interaction occurs for each: 1. Foldable; 2. Bendable; 3. Paper/Cloth; 4. Elastic/Inflatable; 5. Actuated; 6. Liquid; 7. Malleable; 8. Jamming.</figcaption></figure>

## 3.5 Ideation, Elaboration and Storyboarding

Participants were asked to explore the white box prototypes through touch and comparison, then write down ideas for applications, hardware, surfaces or spaces that would benefit or enhance their own lives in some way (e.g. in work, hobbies, social contexts). This was based on the process employed in Sturdee et al.'s public ideation study [56]. Following this, they were asked to choose their favourite idea and expand on it via sketching a diagram and writing notes about – for example – how it works, the user base, interaction, and so on. Finally, the chosen idea was put into context within a storyboarded scenario of use (see Fig. 3).

# 4 Analysis

Fifty participants generated 255 ideas and corresponding sketches for shape-changing interfaces, applications, surfaces and spaces (mean 5.1). They then selected one idea to elaborate upon (n=50). The majority of chosen ideas were indicative of shape-changing hardware (43/50) rather than specific applications for a generic device, although most would allow for multiple applications. Two of the chosen ideas did not specifically address shape-changing technology so were not used in the analysis.

## 4.1 Generating Design Requirements

To elicit requirements from the data, four HCI researchers (to limit bias: one independent from the study, one uninvolved in data collection) coded the data using open coding [53] and affinity diagramming with post-its [33]. Initially, one data set was chosen at random and examined by all researchers, who then generated post-it notes suggesting requirements, interactive properties, context and possible implications for the technology. The group then split into pairs and
worked on the data, and these pairs were rotated. Requirements were extracted in several ways (see Figs. 2. & 3. for an example of participant data): *directly* through notation on the diagrams and storyboards (e.g. device is 20x20cm, device is portable); from examining the *interaction* (device has furry texture, folding and closure must be possible); from the proposed *output* (mimics organic form); and from *context* (device has therapeutic purpose). The *implications* of the technology then arose from the idea of having access to a lifelike, adaptable non-organic representation of a pet – e.g. decline in pet ownership. A guide to the coding and requirements generation is overlaid on Fig. 2.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** Diagram & scenario for *Kitten Everywhere* app, annotated to show how requirements were generated. For example, bottom right – the kitten's paw is overlapping the person's hand and reconnecting with the interface surface – *closure* [47].</figcaption></figure>

Mid-way through the process we examined emergent themes and categories, and it became clear that there was more than simple hardware/software requirements and basic human factors. The post-it notes were then recategorised under specific titles (for example) scale, interaction, portability, multi-sensory, device dependent properties, context of use and so on. The remainder of the initial data analysis was then completed (with further categories emerging) before the next stage, where the clear hardware/software requirements and interaction types (based on [44]), the *physical properties* were temporarily separated off, and the complex, *operational properties* were recoded entirely by the group. Finally, all the categorisations and themes were cross-referenced with the original data and recoded where necessary to create multiple categorical levels, then proofed and the entire dataset digitised and checked for errors. The complete user-generated data set can be downloaded as an appendix.
## 4.2 Results

Analysis of the 255 participant idea sketches produced 506 coded items across three categories – *Requirements* (333), *Applications* (104) and *Implications* (69) – with multiple sub-categories. An item refers to all text-based outputs from the coding process. Items from *Requirements* and *Applications* were synthesised to produce the stack model in Fig. 4. by using top level categories for each and ordering them logically for a prototype build process. From those items falling under the *Requirements* category, we provide those most frequently occurring so as to give an overview of how people appear to think about shape-change (Section 5), and show a categorisation of the top level themes which enables a stacked requirements model (Section 6). Finally, we analyse individual findings and current works to demonstrate the validity of the stack model (Section 7).

# 5 Frequently Occurring Requirements

In total, 333 requirements were generated, across 5 top level categories (Input, Output, Construction & Assembly, Control Systems, and Interactions & Behaviour) which directly relate to the top level headings for the stack model in Fig. 4. The following text contains the highest frequency sub-themes emerging from the analysis (in order of highest frequency), suggesting specific, perhaps essential, requirements for the design of shape-changing interfaces. Examples of sections of original data relating to the categories below are shown in Fig. 3.

**Between Device Communication** (*Interactions & Behaviour*) This finding suggests that our current devices will co-exist with their newer, physically dynamic counterparts. Concepts such as *up-scaling* of film or image from 2D to physical 3D are explored. Shape-change is also expected to communicate *between* devices (e.g. Photo album for the blind – Fig. 3.1).

**Rigidity** (*Construction & Assembly*) Varying the material qualities of a device was communicated by the *jamming* box, and was used in conjunction with other material categories to create behaviours in which a device or application moves between states depending on context of use. This variation is especially important in generic devices where multiple uses are anticipated (e.g. interface which becomes a playable guitar).

**Strength** (*Construction & Assembly*) Shape-change is not only expected to display data, provide comfort or simulate environments, it is expected to be load-bearing in architecture, move boulders with the tap of an application and support multiple bodies as a sofa, car, or podium. To this end, the materials and construction used to create shape-change must be physically robust (e.g. flooring) for the intended application.
<figure><img src="image_4.png" /><figcaption>**Fig. 3.** Examples of shape-changing interface relating to frequently occurring requirements: 1) Photo album for blind; 2) Battle armour; 3) Drinkable Tablet.</figcaption></figure>

**Modularity (Construction & Assembly)** Modularity not only refers to the ability of identical devices to communicate with each other, but for parts of other types of material surface to be removed, used, and reintegrated, or for components of shape-change such as actuators to communicate with each other (e.g. toy blocks).

**Portability (Construction & Assembly)** Many of the participant ideas were categorised as portable object-scale devices, suggesting the need for novel batteries or charging methodologies such as using body movement (kinetic), solar, or wireless. In the requirements, portability emerged as a distinct theme (e.g. armour – Fig. 3.2).

**Multi-sensory Input & Output (Input & Output)** Users are expecting interfaces to have deformation as an interaction technique, and for this technology to also employ the full range of human senses in their application and design, emphasising the organic potential of shape-change (e.g. drinkable computer – Fig. 3.3).

**Organic Movement (Interactions & Behaviour)** By attributing natural and humanistic qualities to range of movement, shape-change crosses into the territory of Artificial Intelligence, or the mimicking of life. Organic movement in
shape-change links to *comfort* and *sensitivity*, reflecting a positive behaviour (e.g. prosthetics).

**Device Personalisation** (*Interactions & Behaviour*) For planar, screen based devices, personalisation usually takes the form of a physical accessory such as a screen protector, or amendments to the display or applications. These amendments can also be attributed to shape-changing devices, but additionally we might control the shape, texture and even how it connects with our bodies (e.g. adaptive training shoes).

## 6 Toward a Design Requirements Model

Our synthesis of requirements of shape-changing interfaces revealed five overarching categories, each at a different level of abstraction. Together, they logically fit into a stacked layer model (Fig. 4) describing levels of requirement and implementation in shape-changing interfaces, and differentiating between physical/operational characteristics. Under the top-level categories, we also identified eight sub-categories of requirements. The top-level categories are outlined below alongside the *Implications* category which was major theme arising from the data not directly connected to design requirements.

<figure><img src="image_5.png" /><figcaption>**Fig. 4.** The stacked model of implementation, based on the requirements generated during the study – see Section 5 for frequently occurring requirements.</figcaption></figure>
## 6.1 Applications & Context

Applications are defined as the specific use envisaged for the device (e.g. battle armour in Fig. 3.3), and “apps” (such as *Kitten Everywhere* in Fig. 2). *Context* applies to where, when, what and why of using the application (e.g. at home, working hours, provide remote massage, travel less for work). Just over a fifth of the items produced during coding were application and context related. Often the context also dictates the application, and this is especially the case for generic, catch-all devices. Poupyrev et al. [42] reviewed potential uses for actuated, tangible interfaces ten years ago and suggested applications and areas based on the literature at that time, creating five categories: *Aesthetics*; *Information Communication*; *Mechanical Work*; *Controls – Data Consistency*; and, *People to People Communication*. In the intervening 10 years however, the range of devices and applications has grown and the five categories remain relevant, but can be blended into the overarching contextual categories generated from more recent work [56]. The application areas generated in this work can be mapped directly onto those found in Sturdee et al. [56] (with the exception of the *infrastructure* category), with some exact application ideas being repeated such as responsive computational flowers, remote massage, or actuated storybooks for children. However we also propose here a wider *generic* category where the properties of the device or surface are used for multiple use-cases.

## 6.2 Interactions & Behaviour

Interaction refers to the relationship the user has with the device and includes the type of interaction, e.g. specific behaviours identified during coding (*squash-to-delete*) but also the interaction the device has with other technology (*between device communication*). *Behaviour* encompasses software actions – what the device does (*switches between planar/3D output*). Eighty-four items fall into this category. Interaction for shape-changing interfaces is perhaps best classified by Rasmussen [44] who developed the framework of *Direct, Indirect and No Interaction*. The requirements for basic interaction in this model also offer other options: *anti-interaction* (the device actively avoids or puts off, interaction), encouraging interaction, and device-to-device communication, covering interaction between shape-changing devices and also between shape-change and current mobile devices and computers. In addition to these high level categories, the behavioural aspects of shape-change are explored with regards to how the device or surface acts or moves to initiate the programmed output, and whether you can personalise your device.

## 6.3 Control Systems

This layer covers the *Software Implementation* for the shape-changing device, outlining *how* the device puts the hardware features into use in programmed, pre-set features and attributes. This is the least dense layer with 20 items, and includes requirements such as *physical shape-memory* and *must have user recognition software*. This can be thought of as the interface operating system.
## 6.4 Construction & Assembly

The physical requirements that are unrelated to sensory input and output are contained in this layer, it contains specific information on hardware and appearance of the device (such as size and portability) as well as non-functional requirements (*washable, lightweight*). It also contains the *Hardware Implementation* which includes information such as *integral camera* or *low latency over internet*. This layer contains over a third of the requirements and over a fifth of the total items.

## 6.5 Input & Output

This layer describes input and output sensors for shape-change, incorporating multi-sensory information in addition to visual and shape output, and specific information such as GPS, speed, texture, temperature and air pressure. Despite a tendency toward two-way multi-sensory interaction for generic shape-changing devices, bi-directionality was not seen as an essential quality for application specific shape-change, which contradicts previous work which suggests this is an overarching feature of tangible interfaces [3].

## 6.6 Implications

Sixty-nine *implications* were generated from the context and use cases implied by the applications. An implication, in this format is a possible direct result or reaction deriving from the adoption and use of a shape-changing technology. The *Implications* were realised both from the application ideas that were generated, but also from specifics within the scenarios and diagrams that the participants created. They are categorised into: *Positive* (of benefit to the majority of the population, such as faster recovery from debilitating injury, improved well-being, and sustainability); *Neutral* (not clearly mapped onto a specific target group or are cannot be categorised, such as “more money for cinemas” or “consequences of shape-changing AI left to own decisions”) and *Negative* (of negative benefit to the majority of the population, such as removing the need for labouring work, or reducing human contact). The possibility of AI taking jobs from people is already a hot topic, and shape-change could enable a far greater obsolescence.

# 7 Using the Requirements Model

We envision that the stack model can be of use for researchers, to be used as a reference list showing the relationship between different parts of the stack, to ensure users consider requirements at all levels (important because the decisions at one layer affect the layers above and below). It could also help categorise and synthesise many requirements together, and function as a tool to direct users to think about certain types/areas of requirements during early design phases. Designers could also use it to ensure they have considered input at all levels (as
incorrect assumptions at one level can propagate), and technologists can use it
to understand the impact of their technology decisions (bottom layers) on the
upper layers. Within the existing framework of shape-change, it could also be
used to assist researchers to better understand existing prototypes – however, we
envisage further development before it can be fully adopted by the community.

## 7.1 Applying the Requirements Model to the Dataset

The overarching requirements and subcategories are distributed across the stack,
with a slight bias toward the multi-sensory, bottom layer of the stack (in-
put/output). Software implementation (*Control Systems*) is the most under-
represented layer, perhaps due to the participant sample we used (non-computer
background), but this omission could be addressed by asking users to specifically
think within the stack system. Looking at the participant data in relation to the
stack model, we could re-analyse the diagrammatical and storyboard data and
identify extra requirements that the user may not have explicitly thought about
during the study. In the case of the *drinkable tablet* (Fig. 3.3) which we know
has *variable rigidity* and is *safe to ingest* we might now return to the under-
represented *software implementation* and extract the information that if it is
a tablet then it also runs apps, and we can see from the sketch that it also
supports a 2D planar screen, meaning it should be backward-compatible. The
demonstrates that the stack model therefore can support directed thinking for
the researchers by identifying specific areas and therefore eliciting further re-
quirements. Observation of the participants during the study suggests that a
mixture of top-down and bottom-up approach is used to create the data: either
where an application idea was realised via consideration of what hardware would
be required to achieve that goal, or beginning with an idea relating to a specific
hardware type based on the white box prototypes, and moving up the stack for
application and contextual design. For example, the *drinkable tablet* idea (Fig.
3.3) came directly from the liquid interface prototype (Fig. 1.6), blended with
the idea of variable rigidity (Fig. 1.8), and although in context of a science fic-
tion story, the practical information remains relatable. Following the ideation
and diagram creation, the context in which a drinkable tablet would be helpful
was then realised – in this case the spy conceals their data by ingesting it – and
a scenario based on this idea was drawn.

## 7.2 Application to Existing Prototypes

We applied a retrospective analysis of the stack model to two existing, contrast-
ing, prototypes to confirm the layers fit with current research, and to get an idea
of where build focus was. The choice of prototype was made by examining the
methodological processes of existing works, focusing on: *Decision-making* – what
was the initial inspiration or background to the work?; *Technology used* – was
it existing, based upon other prototypes, or a novel implementation; *Context* –
what need does the work fulfil?; *Behaviour* – what form of shape-change does
the prototype exhibit?; and, *Inputs/Outputs* – how is the prototype controlled
and what display mechanism does it have? By querying existing research using the stack model as a basis it is possible to retrospectively apply its principles.

<table><caption>Table 1. The stack model for shape-change applied retrospectively to Morpho-Tower [24] and Shape-Clip [20].</caption><thead><tr><th colspan="2">Applications & Context</th><th colspan="2">Applications & Context</th></tr></thead><tbody><tr><td colspan="2">Artistic installation</td><td colspan="2">Rapid-prototyping</td></tr><tr><th colspan="2">Interactions & Behaviour</th><th colspan="2">Interactions & Behaviour</th></tr><tr><td colspan="2">Dynamic reaction to changes in magnetic field, trembling/rotating, defying gravity, increase/decrease in size, organic movement, forms spikes along magnetic field lines</td><td colspan="2">Variable topology, height changes in response to input, reconfigurable</td></tr><tr><th colspan="2">Control systems</th><th colspan="2">Control systems</th></tr><tr><td colspan="2">Variable magnetic fluid</td><td colspan="2">Javascript API, RGB value sampling, supports HTML5, WebSocket-to-Serial bridge, awareness of clip-position</td></tr><tr><th colspan="2">Construction & Assembly</th><th colspan="2">Construction & Assembly</th></tr><tr><td colspan="2">Ferrofluid contained in plate, helical iron tower</td><td colspan="2">Stepper motor, LDRs, RGB LED, ATmega328p, arduino, 3D printed base, circuit board, modular, portable</td></tr><tr><th>Physical Input</th><th>Materiality/Output</th><th>Physical Input</th><th>Materiality/Output</th></tr><tr><td>Magnetic field, electromagnet</td><td>Liquid, dynamic shape, texture</td><td>Light sensor, gesture, force, data</td><td>Actuated pins, form, colour, light</td></tr></tbody></table>

Kodama's ferrofluid works [24] are born of the desire to emulate organic movement and create art, therefore the focus in the stack is on application, and the interactive/behavioural qualities: a top-down perspective (Table 1). Conversely, for *ShapeClip* [20] the focus was on the hardware: building a bidirectional interaction device that was low cost, modular and easy to use (Table 2). This suggests a focus on the lower two layers of the stack (bottom-up), with omissions in detailing interaction behaviours. Both papers refer to future hardware improvements, applications or use-cases (e.g. *advertising*, *sound equaliser*; *third skin*) but do not consider the longer-term implications for their projects (this loosely supports the notion that this research is very technology driven, and not focused on long term adoption [25]). This is not a criticism of existing research, but identifies the requirements focus for different types of shape-change, and where researchers have concentrated their efforts. It also demonstrates the difference between *application* (*Morpho Tower*) and *utility* (*ShapeClip*).

## 7.3 Limitations & Additions

During the coding, it became clear that the aesthetic and emotional aspects of shape-change were largely overlooked, relegated to resulting from – or being incidental to – the device or application itself. Participants tended to focus on the practical aspects of shape-change, which usually started with an application idea (top of the stack) rather than a hardware type, although some items (e.g. shape-changing, responsive flowers) were built with aesthetics in mind, and the non-functional and functional requirements were built around that notion. The lack of focus on aesthetics may suggest two things: That design for aesthetically pleasing objects is a given; or that the desire and design for aesthetics occurs further down the stack – for example, comfort and beauty may be built into the
construction phases of implementation (which makes sense if the purpose of the
device is as a furniture provider). In terms of emotional content or outputs, these
are more likely be an implication of, or bound up in, the type of application (such
as a virtual physical pet to provide company). However, Kwak et al. [27] noted
that the *behaviour* of his prototypes was implied by their actuation, and gave
rise to emotional content e.g. *stubbornness* or feeling *hopeful*. This links to the
*Interactions & Behaviour* category within the stack, within which a number of
the behavioural themes related to organic movement, meaning emotions would
be built in after the software implementation cycle. The stack model could thus
be adapted to add a subcategory of *emotional content*, bound up in the design of
organic movement (one of the minor themes from the coding process), whereas
aesthetics would become a category within *Construction & Assembly*.

8 Discussion

The stack model and analysis suggests that the methodology has the potential
to expand our understanding of how these devices will be built, and applies an
organisational structure to the development process; whereas the implications
generated can stimulate discussion about future adoption of physically dynamic
interfaces. Our methods fulfil their intended purpose: we communicated complex,
novel technology to non-expert end-users who were able to generate detailed out-
puts from which researchers could elicit requirements. The commonly occurring
sub-themes also threw up some novel requirements for shape-change that (to our
knowledge) have not been documented. Between device communication *within*
shape-change is not a new concept, but these requirements also consider inte-
gration of shape-changing interfaces into existing technological structure – e.g.
having an integrated USB port, or sending planar data to be up-scaled. Device
personalisation has been addressed in shape-change, but only in attribution of
types of actuation to different mobile phone notifications [39], when personali-
sation could potentially also involve texture, form, complex organic movement
and multi-sensory experiences – that shape-change should be multi-sensory is
another under-explored facet, especially given that the only essential output is
change in form. Finally, although many of the requirements are pre-existing in
other technology development processes, the way the stack model addresses these
requirements, and how the extra dimensions of movement, organic behaviours
(and so on) are integrated makes the result specific to the application of shape-
changing technology. The fact that we are producing similarities with other work
also means that we are in a good place to begin formalising process and practice
for shape-change – design requirements needed to be approached in a tailored
way to shape-change, and we have appropriately managed this process.

8.1 Methodological Reflection

Due to the range of materials, interaction and applications possible, we did not
use a specific scenario or hardware type (e.g. shape-change for mobile gaming/a
music app for a malleable device). Given the nature of our approach, we felt
that choosing one application or problem would ask the researcher to arbitrar-
ily define that issue, and bias the process toward one kind of shape-change. By
asking users to define their own problem and solution, we explored the nature of
the process in way that still provided focus. During introduction and ideation,
participants found the videos helpful, though two found it difficult to relate what
they had seen to the boxes which did not self-actuate. Others became quite ex-
cited by the boxes, choosing to have one next to them so they could return to a
concept, or taking part of the material (malleable) with them to work through
interactions. The jamming prototype box was seen as the most engaging, as the
material concept was novel and the transition between states illustrated how a
shape-changing interface might move between material properties, but also be-
cause it was in context of the other boxes. Another observation was one of the
hedonic qualities of material interaction [44], some participants enjoyed touching
the materials, and focused upon how the pleasant sensations could be utilised.
Removal of either of the stages would be detrimental, as establishing technologi-
cal context sets the scene and explains what, but providing tangible, low-fidelity
examples encourages participants to ask questions and suggest improvements
about how without focusing on single use-cases.

The sketching and storyboarding process allowed the contextualisation of
ideas in an easily understandable, visual output. Some participants expressed
anxiety during the task, but were reassured by focusing on the ideas rather than
producing high quality artwork. Sketching even had a positive effect: "I haven't
drawn in years, I had forgotten how much fun it was". Given the success engag-
ing with end-users for shape-change, we envisage this process being used in the
early stages of new research. In future work, the technique could be applied using
a single white box prototype, and specific application, and to explore the next
stages of developing shape-changing interfaces – functional prototype develop-
ment/interface design), and the stack model evaluated in the early development
of shape-changing interfaces.

8.2 Implications for Adoption?

As well as approaching user-centred design from the standpoint of shape-changing
interfaces, we are also attempting to consolidate research in this area, and en-
courage the organised advancement of specific interfaces. HCI as a field has
been accused of an unfocused attitude toward research, rarely developing topics
so that they enter the mainstream [25], or criticised for focusing on short-term
utility [32], so by offering up a methodology to engage with possible end users
and suggest constructive avenues to pursue we attempt to counteract this view.
As an extension to this, some researchers suggest reaching even further into
the future to explore not only the adoption of technology, but the implications
of that adoption [31] – how might domesticating technology affect people in
both positive and negative ways? An unexpected but welcome side effect of the
requirements-gathering process was the generation of implications surrounding
the adoption of shape-change. This allows researchers to focus on potentially interesting build-concepts but to also ask *should we*?

## 9 Conclusion

We investigated both the application of visual methods to explore complex technology, and also the value of providing structure to the design and development phase of shape-changing interfaces. Novel requirement themes emerged such as the need for integration into existing technology, device personalisation and between device communication, alongside those already existing in technology. The process used has provided a helpful means of accessing thoughts on pre-existing technology in novice users, and may assist others in researching along these lines by using sketching and lo-fidelity prototyping – capitalising on the engaging and hedonic qualities of shape-change. Not only this, but the formal guidelines for design and development of shape-change could help researchers at the beginning of their work in prototype development. We also imagine that the technique of image generation and analysis requires further work to consolidate it as a viable technique in HCI research, perhaps in the context of other types of prototype or interface design.

Despite the focus of participants on the practical and not aesthetic aspects, aesthetics can be built in by designers and developers once the practicalities of the build are concrete – and the same can apply to *emotionality*. These particular elements of shape-change are evident in other work in the field, e.g. [27, ?], so by integrating our work with that of others in future research, we can formulate a 'world-view' of shape-change – made stronger still by examining existing processes in commercial design and engineering. This may help the field towards adoption rather than becoming trapped in a cycle of rapid development without fixed end-goals. The overall implications of this work relate not only to the eventual adoption of shape-change technology, but how this might change the way in which we consume technology in the future.

To summarise – shape-changing interfaces are complex, emergent technologies for which it is difficult to apply pre-existing processes – to address this, we used non-expert users to generate design requirements for these devices by sketching and diagramming their thoughts after working with video and white-box prototypes. The process produced multiple levels of design requirements which were discerned from open coding of the resulting images. These were categorised and analysed to form a stack model for shape-changing interfaces, which can be applied to future work in platform development. The findings demonstrate new ways of approaching the design of shape-changing interfaces and the continuing development of these highly complex computational experiences. Finally, we also considered the practicalities of adoption, and the long term implications of shape-change.
References

1. Atkinson, D., Orzechowski, P., Petreca, B., Bianchi-Berthouze, N., Watkins, P., Baurley, S., Padilla, S., Chantler, M.: Tactile perceptions of digital textiles: a design research approach. In: Proceedings of the SIGCHI conference on human factors in computing systems. pp. 1669–1678. ACM (2013)

2. Buxton, B.: Sketching user experiences: getting the design right and the right design. Morgan Kaufmann (2010)

3. Coelho, M., Zigelbaum, J.: Shape-changing interfaces. Personal and Ubiquitous Computing **15**(2), 161–173 (2011). https://doi.org/10.1007/s00779-010-0311-y

4. Dand, D., Hemsley, R.: Obake: interactions on a 2.5d elastic display. In: Proceedings of the adjunct publication of the 26th annual ACM symposium on User interface software and technology. pp. 109–110. ACM (2013). https://doi.org/10.1145/2508468.2514734

5. Everitt, A., Alexander, J.: Polysurface: A design approach for rapid prototyping of shape-changing displays using semi-solid surfaces. In: Proceedings of the 2017 Conference on Designing Interactive Systems. pp. 1283–1294. ACM (2017). https://doi.org/10.1145/3064663.3064677

6. Everitt, A., Taher, F., Alexander, J.: Shapecanvas: An exploration of shape-changing content generation by members of the public. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 2778–2782. ACM (2016). https://doi.org/10.1145/2858036.2858316

7. Fish, J., Scrivener, S.: Amplifying the mind's eye: sketching and visual cognition. Leonardo pp. 117–126 (1990)

8. Follmer, S., Leithinger, D., Olwal, A., Cheng, N., Ishii, H.: Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In: Proceedings of the 25th annual ACM symposium on User interface software and technology. pp. 519–528. ACM (2012)

9. Follmer, S., Leithinger, D., Olwal, A., Hogge, A., Ishii, H.: inform: dynamic physical affordances and constraints through shape and object actuation. In: Uist. vol. 13, pp. 417–426 (2013). https://doi.org/10.1145/2501988.2502032

10. Frommer, D.: These are the 10 most popular mobile apps in america. Blog (August 2017), retrieved August 28, 2017 from http://www.recode.net/2017/8/24/16197218/top-10-mobile-apps-2017-comscore-chart-facebook-google

11. Giaccardi, E., Karana, E.: Foundations of materials experience: An approach for hci. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2447–2456. ACM (2015). https://doi.org/10.1145/2702123.2702337

12. Goldschmidt, G.: The dialectics of sketching. Creativity research journal **4**(2), 123–143 (1991). https://doi.org/10.1080/10400419109534381

13. Goldschmidt, G.: Manual sketching: Why is it still relevant? In: The Active Image, pp. 77–97. Springer (2017). https://doi.org/10.1007/97833195646614

14. Gomes, A., Priyadarshana, L., Carrascal, J.P., Vertegaal, R.: Whammyphone: Exploring tangible audio manipulation using bend input on a flexible smartphone. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology. pp. 159–161. ACM (2016). https://doi.org/10.1145/2984751.2985742

15. Gong, J., Li, L., Vogel, D., Yang, X.D.: Cito: An actuated smartwatch for extended interactions. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 5331–5345. ACM (2017)
16. Goulthorpe, M., Burry, M., Dunlop, G.: Aegis hyposurface: The bordering of university and practice. In: Proc. of ACADIA. pp. 344–349. Association for Computer-Aided Design in Architecture (2001)

17. Grönvall, E., Kinch, S., Petersen, M.G., Rasmussen, M.K.: Causing commotion with a shape-changing bench: experiencing shape-changing interfaces in use. In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems. pp. 2559–2568. ACM (2014). https://doi.org/10.1145/2556288.2557360

18. Haesen, M., Van den Bergh, J., Meskens, J., Luyten, K., Degrandsart, S., Demeyer, S., Coninx, K.: Using storyboards to integrate models and informal design knowledge. Model-Driven Development of Advanced User Interfaces pp. 87–106 (2011)

19. Haesen, M., Luyten, K., Coninx, K.: Get your requirements straight: Storyboarding revisited. Human-Computer Interaction–INTERACT 2009 pp. 546–549 (2009)

20. Hardy, J., Weichel, C., Taher, F., Vidler, J., Alexander, J.: Shapeclip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 19–28. ACM (2015). https://doi.org/10.1145/2702123.2702599

21. Hashimoto, S., Suzuki, R., Kamiyama, Y., Inami, M., Igarashi, T.: Lightcloth: senseable illuminating optical fiber cloth for creating interactive surfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 603–606. ACM (2013). https://doi.org/10.1145/2470654.2470739

22. Haughney, E.: Using comics to communicate qualitative user research findings. In: CHI’08 Extended Abstracts on Human Factors in Computing Systems. pp. 2209–2212. ACM (2008). https://doi.org/10.1145/1358628.1358653

23. Ishii, H., Lakatos, D., Bonanni, L., Labrune, J.B.: Radical atoms: beyond tangible bits, toward transformable materials. interactions **19**(1), 38–51 (2012)

24. Kodama, S.: Dynamic ferrofluid sculpture: organic shape-changing art forms. Communications of the ACM **51**(6), 79–81 (2008)

25. Kostakos, V.: The big hole in hci research. Interactions **22**(2), 48–51 (2015)

26. Kotonya, G., Sommerville, I.: Requirements engineering: processes and techniques. Wiley Publishing (1998)

27. Kwak, M., Hornbæk, K., Markopoulos, P., Bruns Alonso, M.: The design space of shape-changing interfaces: a repertory grid study. In: Proceedings of the 2014 conference on Designing interactive systems. pp. 181–190. ACM (2014). https://doi.org/10.1145/2598510.2598573

28. Landay, J.A., Myers, B.A.: Sketching interfaces: Toward more human interface design. Computer **34**(3), 56–64 (2001). https://doi.org/10.1109/2.910894

29. Leithinger, D., Follmer, S., Olwal, A., Ishii, H.: Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In: Proceedings of the 27th annual ACM symposium on User interface software and technology. pp. 461–470. ACM (2014). https://doi.org/10.1145/2642918.2647377

30. Lewis, M.M., Coles-Kemp, L.: Who says personas can’t dance?: the use of comic strips to design information security personas. In: CHI’14 Extended Abstracts on Human Factors in Computing Systems. pp. 2485–2490. ACM (2014)

31. Lindley, J., Coulton, P., Sturdee, M.: Implications for adoption. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 265–277. ACM (2017). https://doi.org/10.1145/3025453.3025742

32. Linehan, C., Kirman, B.J., Reeves, S., Blythe, M.A., Tanenbaum, J.G., Desjardins, A., Wakkary, R.: Alternate endings: using fiction to explore design futures. In: CHI’14 Extended Abstracts on Human Factors in Computing Systems. pp. 45–48. ACM (2014). https://doi.org/10.1145/2559206.2560472
33. Lucero, A.: Using affinity diagrams to evaluate interactive prototypes. In: Human-Computer Interaction. pp. 231–248. Springer (2015)

34. Mackay, W.E., Ratzer, A.V., Janecek, P.: Video artifacts for design: Bridging the gap between abstraction and detail. In: Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques. pp. 72–82. ACM (2000). https://doi.org/10.1145/347642.347666

35. Malafouris, L.: The cognitive basis of material engagement: where brain, body and culture conflate. In: Rethinking materiality: The engagement of mind with the material world, pp. 53–61. Cambridge: McDonald Institute Monographs (2004)

36. Matoba, Y., Sato, T., Takahashi, N., Koike, H.: Claytricsurface: an interactive surface with dynamic softness control capability. In: ACM SIGGRAPH 2012 Emerging Technologies. p. 6. ACM (2012). https://doi.org/10.1145/2343456.2343462

37. Muller, M.J.: Participatory design: the third space in hci. Human-computer interaction: Development process 4235, 165–185 (2003)

38. Nelson, J., Buisine, S., Auoussat, A.: Anticipating the use of future things: Towards a framework for prospective use analysis in innovation design projects. Applied ergonomics 44(6), 948–956 (2013). https://doi.org/10.1016/j.apergo.2013.01.002

39. Park, Y.W., Park, J., Nam, T.J.: The trial of bendi in a coffeehouse: use of a shape-changing device for a tactile-visual phone conversation. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. pp. 2181–2190. ACM (2015). https://doi.org/10.1145/2702123.2702326

40. Pedersen, E.W., Subramanian, S., Hornbæk, K.: Is my phone alive?: a large-scale study of shape change in handheld devices using videos. In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems. pp. 2579–2588. ACM (2014). https://doi.org/10.1145/2556288.2557018

41. Petrelli, D., Soranzo, A., Ciolfi, L., Reidy, J.: Exploring the aesthetics of tangible interaction: experiments on the perception of hybrid objects. In: Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 100–108. ACM (2016). https://doi.org/10.1145/2839462.2839478

42. Poupyrev, I., Nashida, T., Okabe, M.: Actuation and tangible user interfaces: the vaucanson duck, robots, and shape displays. In: Proceedings of the 1st international conference on Tangible and embedded interaction. pp. 205–212. ACM (2007)

43. Ramakers, R., Schöning, J., Luyten, K.: Paddle: highly deformable mobile devices with physical controls. In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems. pp. 2569–2578. ACM (2014)

44. Rasmussen, M.K., Pedersen, E.W., Petersen, M.G., Hornbæk, K.: Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 735–744. ACM (2012). https://doi.org/10.1145/2207676.2207781

45. Rasmussen, M.K., Troiano, G.M., Petersen, M.G., Simonsen, J.G., Hornbæk, K.: Sketching shape-changing interfaces: Exploring vocabulary, metaphors, use, and affordances. In: CHI. pp. 2740–2751 (2016)

46. Read, J.C., Fitton, D., Horton, M.: Theatre, playdoh and comic strips: designing organic user interfaces with young adolescent and teenage participants. Interacting with Computers 25(2), 183–198 (2013). https://doi.org/10.1093/iwc/iws016

47. Roudaut, A., Karnik, A., Löchtefeld, M., Subramanian, S.: Morphees: toward high shape resolution in self-actuated flexible mobile devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 593–602. ACM (2013)

48. Rudd, J., Stern, K., Isensee, S.: Low vs. high-fidelity prototyping debate. interactions 3(1), 76–85 (1996). https://doi.org/10.1145/223500.223514
49. Sahoo, D.R., Hornbæk, K., Subramanian, S.: Tablehop: an actuated fabric display using transparent electrodes. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. pp. 3767–3780. ACM (2016)

50. Schmid, M., Rümelin, S., Richter, H.: Empowering materiality: inspiring the design of tangible interactions. In: Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction. pp. 91–98. ACM (2013)

51. Sefelin, R., Tscheligi, M., Giller, V.: Paper prototyping-what is it good for?: a comparison of paper-and computer-based low-fidelity prototyping. In: CHI’03 extended abstracts on Human factors in computing systems. pp. 778–779. ACM (2003).
https://doi.org/10.1145/765891.765986

52. Sommerville, I., Sawyer, P.: Requirements engineering: a good practice guide. John Wiley & Sons, Inc. (1997)

53. Strauss, A., Corbin, J.M.: Basics of qualitative research: Grounded theory procedures and techniques. Sage Publications, Inc (1990)

54. Strohmeier, P., Burstyn, J., Carrascal, J.P., Levesque, V., Vertegaal, R.: Reflex: A flexible smartphone with active haptic feedback for bend input. In: Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 185–192. ACM (2016)

55. Sturdee, M., Alexander, J.: Analysis and classification of shape-changing interfaces for design and application-based research. ACM Computing Surveys (CSUR) 51(1), 2 (2018). https://doi.org/10.1145/3143559

56. Sturdee, M., Hardy, J., Dunn, N., Alexander, J.: A public ideation of shape-changing applications. In: Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces. pp. 219–228. ACM (2015).
https://doi.org/10.1145/2817721.2817734

57. Sutherland, M., Maiden, N.: Storyboarding requirements. IEEE software 27(6), 9–11 (2010). https://doi.org/10.1109/MS.2010.147

58. Taher, F., Jansen, Y., Woodruff, J., Hardy, J., Hornbæk, K., Alexander, J.: Investigating the use of a dynamic physical bar chart for data exploration and presentation. IEEE transactions on visualization and computer graphics 23(1), 451–460 (2017). https://doi.org/10.1109/TVCG.2016.2598498

59. Wang, J.Y., Ramberg, R., Kuoppala, H.: User participatory sketching: A complementary approach to gather user requirements. In: Proc. of APCHI 2012: The 10th Asia Pacific Conference on Computer Human Interaction. pp. 481–490 (2012)

60. Williams, A.M., Alspaugh, T.A.: Articulating software requirements comic book style. In: Multimedia and Enjoyable Requirements Engineering-Beyond Mere Descriptions and with More Fun and Games, 2008. MERE’08. Third International Workshop on. pp. 4–8. IEEE (2008). https://doi.org/10.1109/MERE.2008.3

61. Winther, M., Vallgårda, A.: A basic form language for shape-changing interfaces. In: Proceedings of the TEI’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 193–201. ACM (2016).
https://doi.org/10.1145/2839462.2839496

62. Ylirisku, S.P., Buur, J.: Designing with Video: Focusing the user-centred design process. Springer Science & Business Media (2007)

63. Zhang Kennedy, L., Chiasson, S., Biddle, R.: The role of instructional design in persuasion: A comics approach for improving cybersecurity. International Journal of Human Computer Interaction 32(3), 215–257 (2016).
https://doi.org/10.1080/10447318.2016.1136177