# Informing Costumers via Interactive Shelves

Peter Rogelj

► To cite this version:

Peter Rogelj. Informing Costumers via Interactive Shelves. 15th Human-Computer Interaction (INTERACT), Sep 2015, Bamberg, Germany. pp.559-562, 10.1007/978-3-319-22723-8_61. hal-01610801

HAL Id: hal-01610801

https://inria.hal.science/hal-01610801

Submitted on 5 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Informing Costumers via Interactive Shelves

Peter Rogelj

University of Primorska, 6000 Koper, Slovenia

peter.rogelj@upr.si

**Abstract.** Consumers often need additional information to decide which products would best suit their needs. This information is in practice limited due to limited space, limits of human attention, large number of products, etc. On the other hand, any approach to provide any kind of information to the customer is effective only if it does not require excessive user involvement. As a solution to offer only relevant information with minimal customer engagement we propose our vision of interactive shelves. The general idea is to observe the customer interaction with the products in order to recognize and display relevant information. At the moment the observation of interaction is achieved by using passive infrared sensors (PIR) and ultrasound distance measuring sensors (US) to detect user grabbing or pointing to products. We are planning to enhance the current system with camera and gaze detection in the future.

**Keywords.** Interest recognition; shopping shelves; sensor integration.

## 1 Introduction

In a process of selecting/buying products, customers need to take several decisions based on the information available. Traditionally they get support from shop assistants, which can advise them and provide additional information. Nevertheless, it is important that customers are able to get certain information of the products even without such assistance. One way of providing information is by writing it on declaration labels of products, which are usually limited due to space constraints. In addition, the process of informing is affected by the large number of products available and human ability to extract the required information. All these adds to the amount of effort required for the costumer to extract relevant information and reading product declarations often turns out to be too complex to be effective.

The described limitations can be solved with interactive shelves that would recognise costumers' information needs with minimal customer engagement. As a good shop assistant, the interactive shelves would need to provide only that much information the customers can bear and only the information that is relevant for the specific customer at the specific point in time. Information selection must be thus based on customer interaction with the environment. Our solution to recognise costumers' interaction include the detection of grabbed or pointed products as well as detection of their gaze.
## 2 System description

Interactive shelves offer customers additional information on display(s) mounted in the vicinity of the products, usually above or on the side of the shelves. The shelves are equipped with sensors that detect presence and position of customer hands when reaching towards the products. The combination of passive infrared sensors (PIR) used for presence detection and ultrasonic distance sensors for accurate hand position detection is proposed. The information of hand location is used to recognize customer's information requirements. We see additional possibilities of customer interest detection using camera(s) and gaze detection, omitting the need of any kind of active customer involvement. A possible configuration of shelves equipped with display and sensors is illustrated in Figure 1. In practice, all the sensors are small and hidden. The detected interest/information requirements are used to select predefined information pages to be displayed on display(s). In the initial implementation each product is explained with one information page displayed at the time of detected customer attention to the product. In addition, product observation history could also be used to make recommendations of related products. Each information page may consist of any type of information, e.g., text, images, videos or their combination.

<figure><img src="image_2.png" /><figcaption>Fig. 1. A typical sensor configuration of interactive shelves. In practice, all the sensors are small and hidden to the customers.</figcaption></figure>

## 3 System design and implementation

One of the primary goals of the system design was to make it modular, such that additional advanced functionality can be added to the initial basic functionality. Thus, the system is designed to be easy to upgrade and also easy to understand. The system architecture, consisting of five layers and a data storage, is illustrated in Fig.2.
<figure><img src="image_3.png" /><figcaption>Fig. 2. Interactive shelves system architecture.</figcaption></figure>

**Physical layer.** Physical layer consists of hardware sensors. In our implementation they are all connected to the main processing unit, the Raspberry PI version 2. Due to relatively large number of PIR and US sensors they cannot all be directly connected to the GPIO port such that I2C IO extensions (using integrated circuit MCP23017) are planned. For future upgrades additional sensors are possible, e.g., camera(s) for gaze detection [1].

**Localization layer.** It is used to process the signals from the sensors in order to obtain information of the location of interest, i.e., in the basic implementation the coordinates of detected customer hands reaching towards the shelves. This layer requires additional information of sensor geometric configuration, which depends on actual shelves geometry. For a typical sensor configuration see Fig.1. Information from all the sensors is integrated into a single vector of interest location.

**Product identification layer.** It serves to identify which product is in focus of customer interest. It requires information of product locations on the shelves, which it gets from the system data storage. Typically the closest products are selected.

**Information strategy layer.** It is used to define a strategy of displaying additional product information on display(s). Its implementation is trivial for the basic functionality where only one page of additional information is planned for each item. Its advanced version may consist of two stages, i.e., interest recognition obtained from
metadata of few latest products and definition of a sequence of information pages providing additional information for detected interest. The decision of sequence of information pages shall be defined according to metadata of products, interests and available information pages, similar as it was developed for e-shopping [2]. The result of this layer is an information page ID to be displayed at the moment (or multiple IDs if multiple displays are used).

*Presentation layer.* The additional information regarding products on shelves is displayed using a standard web browser in full screen mode. All the pages are prepared in advance and passed from the system data storage over a standard HTTP server. A simple Javascript function running in a browser periodically checks for changes of page ID by calling a simple PHP script on the server. Whenever the page ID changes, it reloads the page in the browser’s main frame. This approach enables pages to consist of any kind of information, not only static text and images, but also videos or other dynamic content.

*Data storage.* There are two main entity types stored in system data storage, i.e., products and information pages. They together present all the information specific for the interactive shell usage and can be administered from a dedicated web interface. Both entity types have attributes that relate them with customers’ interests and enable their linkage in the information strategy layer.

# 4 Conclusion

The interactive shelves are a step towards improved customer experience. The actual system is currently in the implementation stage. In addition to technical implementation issues explained here, other questions need to be considered before using such technology in practice. For example, the specifics of customer psychology and marketing issues shall be clarified in order to find the best positioning of displays, temporal specifics of interest recognition and sequence of information pages, recommend principles of product ordering, and to test different design issues of information pages. The concept could be extended to general information points, which could in addition to shelves have a form of interactive boards, posters or maps. As such the concept has a wide usage not limited to product shopping but also to tourism and services.

# 5 References

1. Dostal J, Hinrichs U, Kristensson PO, Quigley A.: SpiderEyes: Designing Attention- and Proximity-Aware Collaborative Interfaces for Wall-Sized Displays. Proceedings of the IUI 2014.

2. Kwon OB: "I know what you need to buy": context-aware multimedia-based recommendation system. Expert Systems with Applications 25 (2003) 387-400.