# Smart Interactive Packaging as a Cyber-Physical Agent in the Interaction Design Theory: A Novel User Interface

Justina Lydekaityte

► To cite this version:

Justina Lydekaityte. Smart Interactive Packaging as a Cyber-Physical Agent in the Interaction Design Theory: A Novel User Interface. 17th IFIP Conference on Human-Computer Interaction (INTER-ACT), Sep 2019, Paphos, Cyprus. pp.687-695, 10.1007/978-3-030-29381-9_41. hal-02544584

HAL Id: hal-02544584

https://inria.hal.science/hal-02544584

Submitted on 16 Apr 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Smart Interactive Packaging as a Cyber-Physical Agent in the Interaction Design Theory: A Novel User Interface

Justina Lydekaityte

Aarhus University, Birk Centerpark 15, 7400 Herning, Denmark
justina@btech.au.dk

**Abstract.** The emerging infrastructure of cyber-physical systems consisting of everyday items, as product's packaging, and advanced digital communication devices opens a new digital dimension for interaction and user experience. Consequently, the concept of human-packaging interaction goes beyond the pragmatic aspects of physical packaging attributes, and, in turn, embraces the potentials of ICT systems. Due to the new forms of human-packaging interactive systems, designers have to address the relevancy of the interaction design and the complexity in the relationship between consumer behavior and interactive system design, i.e. digitally-enhanced packaging. Therefore this research aims to describe the digitally-enhanced packaging as a digital interactive system in regards to the theories of human-computer interaction, interaction design, and user-centered design. In this paper, the critical elements of the interactive packaging design are described. This study concludes that for the interactive systems to be effective and used, designers have to build not only a reward-based, intuitive and simple interaction design that would persuade users to take actions, but also they have to think of other mediate interactions, internal and external resources that are significant to reach the final aim.

**Keywords:** Human-Packaging Interaction, Smart Interactive Packaging, Interaction Design, Cyber-Technical Systems.

## 1 Introduction

Within the exponential growth in the application of computer systems, a wide range of all sorts of artifacts and interfaces have arisen ranging from mobile devices and domestic appliances to vehicles and whole houses [1]. The given access to these gadgets by information and communication technology (ICT) made communication more advanced and diverse. Today the consumer market brings into play many miscellaneous digital interfaces to create the interaction between consumers, products and brands to deliver unexpected and unique experiences [1]. Product packaging also became one of such digital interfaces.

Out of the many roles packaging has to perform, the user interaction is likely to have a profound effect on packaging innovation [2]. Packaging, also referred as ‘communication surface’, ‘an extended user interface’, ‘communication medium’, ‘contact point’,
or ‘silent salesman’ encounters consumers daily through various visual and tactile in-
terplays [3]. The communication between the user and packaging occurs in every step
of the supply chain, including producer, distributor, retailer and end consumer [4]. The
current research on human-packaging interaction (HPI) [5, 6], also called as user-pack-
aging interaction [4] or consumer-packaging interaction [3], investigates either the er-
gonomic- or marketing-concerning factors. The former is related to the handling and
usability of packaging and the utilization of the packed product [6], whereas the latter
is associated with the visual appearance of the packaging [4] in terms of physical at-
tributes as color, shape, material or typography. However, the emerging infrastructure
of cyber-physical systems, induced by advanced wireless communication devices and
IoT, opens a new digital dimension for interaction and user experience [7] and goes
beyond the pragmatic aspects of HPI. Consequently, traditional passive packaging is
able to embrace the digital transformation and become network-connected [8] due to
applied a wide range of mobile, digital and wireless communication technologies. As
these technologies improve, the new forms of human-packaging interactive systems
appear, and thereby designers have to address the relevancy of the interaction design in
relation to the complex relationship between the consumer behavior and interactive
system design, i.e. digitally-enhanced packaging [9]. Therefore this research aims to (i)
describe the digitally-enhanced packaging as a digital interactive system in relation to
the theories of human-computer interaction, interaction design, and user-centered de-
sign, and, in turn, (ii) investigate what are the key elements of designing an effective
interactive packaging design.

This research presents the work in progress and it is based on literature review fo-
cused on articles related to human-packaging interaction, HCI, interaction design, and
user-centered design. This study creates a link between the everyday item, as product's
packaging, and interaction design of HCI systems.

2 Theory

## 2.1 Smart Interactive Packaging

Generally, product packaging can be perceived as: “a socio-scientific discipline which operates in society to ensure the delivery of goods to the ultimate consumer” [5]. It is also defined as a combination of product, package, and distribution which is intended to provide the functions of protection, convenience, containment, and communication [4, 10]. However, the importance of the packaging role and the improvement of its functionalities have increased over the years due to changes in market globalization, demographics, lifestyles and consumer preferences [11]. Having in mind that packag- ing already served as an effective means of communication medium [10], recent ad- vances in printed electronics, conductive printed materials, and wireless communica- tion devices improved the communication function even more. This transformation al- lowed packaging to enter digital innovation and become network-connected [8]. As a result, smart interactive packaging goes beyond the traditional one-way informational flow and triggers the unique interaction capability between the package and consumer.
Reference [12] contributes and states that integrated printable circuits onto consumer packaging would add to products such features as brand protection, customer feedback and visual product enhancement. Connected packaging ability to collect and analyze data empowers brands to understand the effectiveness of the packaging/product and consumers' engagement better, and dynamically adapt to emerging needs by improving their services and products. Therefore, the design of smart interactive packaging, as an interactive system, has to take into account both the insights of the interactive design and user experience.

## 2.2 Interaction Design of Digitally Enhanced Packaging

With the increasing use of the Internet, home and leisure computing, and digital interactive consumer products, the two disciplines of engineering and design merged due to a common goal to amplify discretionary use and user experience [13]. The perspective of the user and the context of use went beyond the traditional computer and mechanical systems and started to penetrate into products and environments people interact with in daily life [5]. Consequently, the user-centered design has broadened to other cultures that gave new opportunities for consumer industry and brand owners.

The popularization of consumer-oriented ICT systems lets designers create special moments and environments giving brands the opportunity to have in-depth communication with their consumers filled with emotional and sensorial facets. As a result, it forms an exceptional link between customers and manufacturers [1]. However, the success of creating this bond depends on whether the designed artefacts and environments can offer a pleasing interface with the user [1]. Therefore, the main aim of interaction design is to: "create interactive products and systems which are usable – easy to learn, effective and pleasant to use" [1]. Reference [14] concurs and states that interaction design has been aroused by increasing industry's demand for intuitive, effortless and enjoyable computing systems. User-centered design is intended to transfer user needs into products specifications to ensure the satisfaction aspect [5]. Generally, the interaction design combines elements of HCI and user experience design to build overall essence and structure of interactive systems that support and facilitate user's goals for helpful and engaging product interfaces [1, 14]. In other words, interaction design concentrates on constructing the ways users interact with products and systems.

The design of digitally-enhanced packaging, as a digital interactive system, has to follow the principles of interaction design, if the functional and pleasing user experience is the main goal to accomplish [5]. There are four critical elements for enhanced packaging design that is based on principles of user-centered design, consumer experience, HCI, and usability theory presented by [15]: consumer, task, package and context.

### Understanding the Consumer

Since consumer experience plays a central role in the interactive packaging design, the investigation of (i) the characteristics of the person including physical and cognitive capabilities, beliefs, habits and previous experience, as well as (ii) the way people respond to a stimulus is needed [5, 15]. Also, it is relevant to understand the user's needs and desires in a thorough manner to design interactive solutions that address these needs precisely [16]. Furthermore, the user's perception is built during the interaction with
packaging [4]. Once an interactive system is able to find the best way to engage with its user, stronger emotional and memorable reactions are provoked that might result in higher efficiency and recurrent use of the interactive system [17].

## The Task

Interactive packaging design has to consider the series of actions and goals to be performed and accomplished by the user that interacts with the package [15]. In regards to HCI, these tasks go further from conventional actions carried out with packaging as opening, handling, reading instructions, disposing [4], and involve a new set of interactive activities related to ICT systems, where users, for example, have to bring their mobile devices to scan the package, download an app, enabling specific communication settings to enable the consumer-packaging interaction [18].

## The Package

Digitally enhanced packaging can be embodied with various digital communication electronics, thus the design and integration of these objects of interaction should also be taken into account. Reference [7] refers to such packaging as a hybrid digital physical object consisting of Cyber-Physical Systems, Cloud Computing and IoT. Cyber-Physical Systems, like microprocessors, sensors and actuators can be embedded into objects, like product's packaging, and the interaction will happen not directly with the digital device, but with ordinary everyday objects with concealed digital technology [7].

## The Context

Another critical element of HPI is the identification of specific stages, context or touch-points, where users interact with packaging in order to support designers and manufacturers and help them understand the elements necessary at each stage of interaction to evaluate, modify or develop packages that would achieve targeted goals [4]. The first stage of the interaction occurs in the distribution system, including warehousing, transportation, and stacking. Due to ICT, packaging with integrated RFID tags can improve real-time location tracking and, in turn, ease logistics operations, whereas packaging with smart temperature, pressure, or shock sensors can register accidents during distribution and handling allowing users to re-evaluate the most efficient means of transportation and the best conditions for it [8]. The second stage takes place at the point of purchase, where packaging reaches the retailer and thereby it has to fulfill a set of new communication activities to draw attention, convince or persuade consumers to purchase the product [2]. At this stage, for instance, light emitting devices or capacitive touch sensors added to the package's exterior design provide distinctive characters as flashy and multisensory effects that may add value to the product and trigger momentary and instantaneous desire to purchase it due to peculiar visual appearance [17, 18]. Finally, once the packaging is bought, it lives at consumer's home and becomes a part of their life, therefore more tactile-based in-depth interaction happens during consumption and utilization of the packed product [3]. Contrary to visual awareness, usage after purchase might give an impulse to the emotional and physical connections to the product and the brand [17]. Sensory, emotional and social sensations induced by IoT-enhanced packaging providing insights of user consumption behavior to improve his/her health condition can be the building blocks for better engagement and entertainment. As a result, this research put emphasis on the last two stages of consumer-packaging
interaction, in-store and at-home, due to their particular importance for user-centered design, user experience and interactive activities.

# 3 Cases of Smart Interactive Packaging

In this section three conceptual cases of digitally enhanced packaging will be described including emphasizing the perspective of user-centered design, instead of technological capabilities of the interactive system. The described cases will be used for more in-depth assessment in regards to the HCI and interaction design in the discussion. Furthermore, each case is summarized in Table 1 according to the four-elements-based framework of user-centered and HCI-supporting packaging design presented in the theory section.

## 3.1 Olive Oil Package with Attached NFC Tag

The credibility of the source the product was obtained from could have a higher impact on persuading the consumers to purchase a product [18]. Therefore the interactive visual demonstration of the origin of the food product, as olive oil, as well as the conditions and the environment of the plants and harvest might trigger instant decision to buy the product. Cyber-Physical Systems can bring the consumers during their grocery shopping closer to the olive tree plants in sunny southern Italy. The olive oil packaging with incorporated NFC tag can redirect the shopper to a website of the olive oil producer filled with photos and videos of the farming site by a single scan on the package with a mobile device.

## 3.2 Cereal Package with Integrated NFC Tag

Although marketing is considered as a secondary function of packaging, in the retail environment it plays a significant role in convincing shoppers to take the item out of the shelf and place it in their shopping bag [4]. One of the highly persuading marketing techniques is the coupon, voucher or discount system. A cereal package with an advert "tap me with your phone and get 10% to milk", for instance, in exchange of the email address, would give higher changes that the products will be bought since cereals and milk are usually consumed together. Likewise, by tapping on a NFC tag attached to cereal packaging, users can download a discount code or voucher valid for a particular period.

## 3.3 Mouthwash Bottle with Smart Sensors

As mentioned earlier, at the stage of product usage it is more likely to make strong emotional, sensory and social connections to the product and brand. However, more pretentious aims require higher consumer interaction resulting in continuous and long-lasting activities/tasks. Likewise, the more time the activities take, the more sophisticated ICT systems are enrolled in the overall interactive system design. In this case,
Cyber-Physical Systems, as smart capacity sensors, Cloud Computing and IoT cooperate for better engagement [7]. A smart capacity sensor incorporated in the mouthwash lid can estimate how much of the product is left, then collect, transmit and analyze the data to build a personal profile for a user to track his/her usage history and dental hygiene habits. The interactive system can contribute to the user's well-being and encourage healthier behavior in a form of reminders.

<table><caption>**Table 1.** The summary of each packaging case</caption><thead><tr><th>Package</th><th>Users</th><th>Tasks (few examples)</th><th>Context</th></tr></thead><tbody><tr><td>Olive oil package with attached NFC tag</td><td>Grocery shoppers<br>Olive oil users</td><td>Enable NFC settings<br>Download the app (iOS)</td><td>In-store</td></tr><tr><td>Cereal package with attached NFC tag</td><td>Grocery shoppers<br>Cereals users</td><td>Find symbol and scan/tap</td><td>In-store</td></tr><tr><td>Mouthwash bottle with smart sensors in the lid</td><td>Dental hygiene supporters</td><td>Download the app<br>Consume the product<br>Track personal profile<br>React to reminders</td><td>At-home</td></tr></tbody></table>

# 4 Discussion & Conclusion

The aim of this section is twofold. The first part will describe and illustrate how digitally enhanced packaging as a digital interactive system fits the overall HCI and interaction design theory. The second part will present the five steps approach that should be considered when designing a successful interaction packaging design.

## 4.1 The Design of Digitally Enhanced Packaging

In relation to the interaction design theory presented by [19], the design of smart interactive packaging usually encompasses (i) the human agent, i.e. the consumer of the product, (ii) the computational agent, i.e. the mobile device, and (iii) the cyber-physical agent consisting of a physical product packaging and digital communication devices (Figure 1). In this model of interaction, the computational agent is an intermediate part between the human agent and cyber-physical agent. In other words, the interaction between the human agent and the cyber-physical agent can only be granted by the computational agent. For instance, in the presented case of olive oil packaging, the shopper first has to interact with the mobile device (to download the app, enable settings, unlock the screen, and other), and only then tap with the device of the package.

On the other hand, this sequence of interactions and the involvement of different agents highly depends on the ICT system incorporated into packaging design and could be done the other way around, i.e. human agent-package-mobile device. For example,
packaging with printed capacitive touch buttons will induce direct human agent-pack-
aging interaction, and a mobile device could be used to display the digital content
aroused by this interaction.

<figure><img src="image_2.png" /><figcaption>Figure 1. The interacting agents in the interaction design of digitally-enhanced packaging</figcaption></figure>

## 4.2 The Five Step Approach for Interactive Packaging Design

There are five critical concerns that should be addressed when designing a successful interaction packaging design:

1. Why the user should take action or perform a task?

First designers should think carefully how to encourage consumers to use technolo-
gies [18]. Because it is a consumer that chooses to download or open a mobile ap-
plication or not in order to obtain digital packaging experience [18]. In this stage,
according to [18]:"marketers must first convince the consumer to use their applica-
tion before convincing them to buy their product". Therefore, in order to take action,
consumers should get a stimulus from the environment [1], an implied benefit upon
completion [19] in the form of a particular reward. Consequently, if the user is sat-
isfied with the reward, it can contribute to continued and enhanced usage of the in-
teractive system [9].

2. *Is the overall design intuitive and simple to use?*

For successful implementation and acceptance of a system, users have to be con-
sciously aware what actions to take [19, 20]. For instance, the graphic design of the
interactive system must clearly state where to scan or tap with the phone, which
mobile application to download and etc. Also, actions have to be simple and intui-
tive, because the design of any interaction has to consider the human agent's inherent
capacity to accomplish this task [19]. As a result, the designers have to build simple,
fast and intuitive actions that could be carried out without mastering any extra skills
[20].

3. What other interaction might appear in the process of accomplishing the main in-
teraction?

The modelling of the interaction space that surrounds the new interaction designers
wish to create is significant and consists of the two main steps [19]. First, other
agents that will be local to the new interaction have to be indicated, and then their
likely effects on the new interaction have to be examined [19]. The comprehensive
analysis of other mediated interactions enhances the chance for the main interaction
to succeed [19]. In the context of the interactive packaging system, the main inter-
action is between the consumer and product packaging. However, other forms of
mediate interactions, such as user-phone or phone-packaging, have to be taken into
account in order not to subvert the main interaction. As a result, it is crucial to keep
the user motivated during all steps of interaction to reach the final aim [19].

4. What other internal and external resources are needed for accomplishing the inter-action?

Designers have to take into account and build all internal and external ICT systems that support and are directly related to the core interaction. In terms of smart inter-active packaging, the internal resource could be a mobile phone that enables the user to perform a task, i.e. tap on the package and read the NFC tag. Whereas, the external resource could be a QR code printed on the package to download the app for NFC tag reading. Also, one should consider that such internal or external agents have their own tasks, cost, benefit, and limitations [19]. Therefore, the implied benefit upon the completion has to be greater than the cost of resources in order to induce the human agent that all actions are worth doing [19].

5. What other attitudes, intentions and motivations of user have to be incorporated into overall design?

The design of information and communication system has to consider the people who will use them [19]. User-centered design demonstrates a great importance in the design process, thus designers have to investigate their potential users attitudes, intentions, motivations, and inspirations [19]. According to the author, the user research with the aim to ascertain their goals has to be carried out before creating interactions.

Based on the findings, it is apparent that for the interactive systems to be effective and used, designers have to build not only a reward-based, intuitive and simple inter-action design that would persuade users to take actions, but also they have to think of other mediate interactions, internal and external resources that are significant to reach the final aim. New insights of consumer packaging as a digital interactive system are expected to have significant practical implications for brand owners and retailers that aim to improve their consumer engagement and make memorable, long-lasting connec-tions. Especially, in these days, when people are always connected to the Internet, new forms of interaction with purchased goods via ICT technologies might turn into unique business models to improve consumer satisfaction, perception, and loyalty.

5 References

1. Bezerra, P. F., Arruda, A., Araujo, K.: Experience Design as a tool to promote interaction among users in the beverage market: proposal for a new emotional approach in usability. Procedia Manufacturing 3, 6028-6035 (2015).

2. Wever, R.: Touching tubs and grabbing gable-tops: an editorial to the special issue on human-packaging interaction. Packaging Technology and Science 29(12), 603-606 (2016).

3. Ryynänen, T., Rusko, E.: Professionals' view of consumers' packaging interactions—a narrative analysis. Packaging Technology and Science 28(4), 341-355 (2015).
4. Mumani, A., Stone, R.: State of the art of user packaging interaction (UPI). Packaging Technology and Science 31(6), 401-419 (2018).

5. Carli Lorenzini, G., Olsson, A.: Towards patient-centered packaging design: An industry perspective on processes, functions, and constraints. Packaging Technology and Science 32(2), 59-73 (2019).

6. Joutsela, M., Latvala, T., Roto, V.: Influence of packaging interaction experience on willingness to pay. Packaging Technology and Science 30(8), 505-523 (2017).

7. Petrelli, D.: Industry 4.0: Is it time for interaction design craftsmanship? The Design Journal 20(sup1), S2735-S2745 (2017).

8. Nilsson, H. E., Unander, T., Siden, J., Andersson, H., Manuilskiy, A., Hummelgard, M., and Gulliksson, M.: System integration of electronic functions in smart packaging applications. IEEE Transactions on Components, Packaging and Manufacturing Technology 2(10), 1723-1734 (2012).

9. Candy, L., Costello, B.: Interaction design and creative practice. Design Studies 6(29), 521-524 (2008).

10. Schaefer, D., Cheung, W. M.: Smart Packaging: Opportunities and Challenges. Procedia CIRP 72, 1022-1027 (2018).

11. Azzi, A., Battini, D., Persona, A., Sgarbossa, F.: Packaging design: general framework and research agenda. Packaging Technology and Science 25(8), 435-456 (2012).

12. Tudor Gethin, D., Huw Jewell, E., Charles Claypole, T.: Printed silver circuits for FMCG packaging. Circuit World 39(4), 188-194 (2013).

13. Lowgren, J. The Encyclopedia of Human-Computer Interaction, https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/interaction-design-brief-intro, last accessed 2019/05/27.

14. Wray, T. B., Kahler, C. W., Simpanen, E. M., Operario, D.: User-centered, interaction design research approaches to inform the development of health risk behavior intervention technologies. Internet interventions 15, 1-9 (2018).

15. de la Fuente, J., Bix, L.: A tool for designing and evaluating packaging for healthcare products. Patient Compliance 1, 48-52 (2011).

16. Stolterman, E., Wiberg, M.: Concept-driven interaction design research. Human-Computer Interaction 25(2), 95-118 (2010).

17. Tafesse, W.: An experiential model of consumer engagement in social media. Journal of Product & Brand Management 25(5), 424-434 (2016).

18. Petit, O., Velasco, C., Spence, C.: Multisensory consumer-packaging interaction (CPI): The role of new technologies. In: Multisensory Packaging, pp. 349-374. Palgrave Macmillan, Cham (2019).

19. Coiera, E.: Interaction design theory. International journal of medical informatics 69(2-3), 205-222 (2003).

20. Maguire, M.: Socio-technical systems and interaction design–21st century relevance. Applied ergonomics 45(2), 162-170 (2014).