# Zapzap: A Table-Top Device that Presents the Rough Contents of a Book to Support Book Browsing in Bookstore

Masahumi Muta, Soh Masuko

<figure><img src="image_1.png" /></figure>

► To cite this version:

Masahumi Muta, Soh Masuko. Zapzap: A Table-Top Device that Presents the Rough Contents of a Book to Support Book Browsing in Bookstore. 16th International Conference on Entertainment Computing (ICEC), Sep 2017, Tsukuba City, Japan. pp.447-451, 10.1007/978-3-319-66715-7_60 hal-01771260

HAL Id: hal-01771260

https://inria.hal.science/hal-01771260

Submitted on 19 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Zapzap: A Table-Top Device that Presents the Rough Contents of a Book to Support Book Browsing in Bookstore

Masahumi Muta* [0000-0001-5795-0603] and Soh Masuko* [0000-0002-3277-5690]

Rakuten Institute of Technology, Rakuten, Inc.,
Rakuten Crimson House, 1-14-1 Tamagawa, Setagaya-ku, Tokyo, Japan.
{masafumi.muta,so.masuko}@rakuten.com

**Abstract.** In this paper, we present the concept, prototype design and implementation of Zapzap, a table-top device that shows the rough contents of a book placed on it. Zapzap is expected to be deployed in bookstores to support customers browsing books. When the user places a book on the display, the system shows keywords, i.e., words often included in the book. When the user taps one of the displayed words, a sentence that includes the selected word is shown. Zapzap aims to present the atmosphere of a book without spoiling its story.

**Keywords:** Table-top, Book, E-book, Shopping, Entertainment computing

## 1 Introduction

<figure><img src="image_3.png" /><figcaption>Fig. 1. A user using Zapzap.</figcaption></figure>

With the rise of electronic books (e-books) and the Internet bookstore such as Amazon, the number of brick and mortar bookstores is decreasing [4]. The merit of using an Internet bookstore is that we can easily order books that we have already decided to buy. However, physical bookstores provide us with the possibility of discovering new books. Therefore, we believe that Internet
bookstores and traditional bookstores should coexist. In this paper, we propose Zapzap, which supports a user in discovering new books in a bookstore, by presenting the rough contents of a book, thereby enticing the user into buying the book.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Schematic of Zapzap.</figcaption></figure>

## 2 System Overview

Zapzap is a table-top device and comprises a RGB-D sensor and two displays, called the top display and the side display. The top display includes a touch sensor. We used ELO 3230L touchable monitor for the top display, a Philips BDM3201FC/11 monitor for the side display, and Intel Realsense F200 camera for the RGB-D sensor. Figure 2 shows a schematic of the device.

Once the user places a book on the top display, the system recognizes the book via the RGB-D sensor and then displays keywords, i.e., words often included in the story, around the book. Simultaneously, the side display shows the blurred cover of the book so that users know that the book has been recognized by the system. The user can tap any of the words shown on the top display. On tapping, the system shows a random sentence that contains the keyword.

### 2.1 Book Recognition and Keyword Positioning

To display the keywords of the placed book, we need to identify which book is on the table. It is also important to track its location to layout the keywords around the book. We used AlexNet [2] to identify the book cover. To prevent flickering due to temporal misrecognition, we designed the system to accept the output from AlexNet only when it has recognized the same book for three consecutive frames.

For book tracking, we used the depth image obtained from the RGB-D sensor. As the sensor is placed on the display, the surface of the book is closer to the sensor compared with the display surface; therefore, we can assume the centroid of the closer pixels as the center of the book. We only extract pixels that are closer to the sensor but limit extraction within 3 cm of the top display so that the user's hand can be maintained over the display without affecting tracking.
## 2.2 Extracting Keywords

We extract the keywords from the full text of the electronic version of the book. We morphologically analyze the full text and then pick the top ten common and proper nouns. We show an example of the extracted words as follows. The words with (n) are character names, (l) indicates a location, and (b) indicates a brand. This method tended to preferentially extract character names.

**Ikebukuro West Gate Park<fnref n="1" />**

Kana (n), Ikebukuro (l), Makoto (n), Takashi (n), Shun (n), Kyoichi (n), Isogai (n), Yamai (n), Kazunori (n), Kenji (n)

**Mr. Mercedes (part 1)<fnref n="2" />**

Brady (n), Mercedes (n/b), Pete (n), Paula (n), Handley (n), Barbara (n), Frankie (n), America (l), Toyota (b)

## 2.3 Placing Keywords

<figure><img src="image_5.png" /><figcaption>Fig. 3. Examples of keyword layouts.</figcaption></figure>

When displaying the keywords, we decided their placement by applying a force directed graph [1]. We considered a graph with all the words and a virtual middle point, which represents the position of the book, as its nodes. Then, we set the links between all the nodes. We set the strength of the links to $K_{ww} > K_{ww} > K_{wc}$, where $K_{ww}$ is the strength of the links between each word and $K_{wc}$ is for the links between each word and the virtual middle point. This causes the words to be arranged such that they spread around the middle point. In addition, we added gravity, which forces the words to gather around the center

<footnote n="1">¹Written by Ira Ishida, Bungeishunju Ltd.</footnote>

<footnote n="2">²Written by Stephen King, Translated by Rou Shiraishi, Bungeishunju Ltd.</footnote>
of the display so that the words is not placed outside the display even when the
book is placed on the corner. Note that we indicate the virtual middle point to
be where the book is placed to help with understanding, even though the book
is not actually visible.

Figure 3 shows an example of the layouts. Figure 3a shows the layout when the user places the book to the center of the display, while Figure 3b shows the layout when they place it at a corner of the display.

3 Related Work

Murai and Ushiama proposed an interface to support the effective browsing of e-
books by users [3]. Their system displays an attractiveness map, which visualizes
the estimated transition of a user's interest through the story of the book, thereby
allowing users to know where to first look in the book. While their interface aims
to be used in a computer display to browse e-books, our interface aims to be
used in physical book stores to browse actual books.

4 Conclusion and Future Work

We presented the concept, prototype design and implementation of Zapzap, a
table-top device that shows the rough contents of a book to support a user
browsing books in a physical bookstore. The system recognizes the book and then
presents the content as keywords and sentences to avoid ruining the entire story.
In a future study, we plan to extract more attractive keywords compared to the
current method, i.e., the most frequent words. To achieve this, the attractiveness
map proposed in Ref. [3] may be applicable. Moreover we plan to show additional
information gathered outside the book such as the author information or online
reviews.

After making these improvements, we will conduct an experiment to deter-
mine how informative the system is for the user and whether it motivates the
user to read or purchase the books.

References

1. Dwyer, T.: Scalable, versatile and simple constrained graph layout. In: Proceedings of the 11th Eurographics / IEEE - VGTC Conference on Visualization. pp. 991–1006 (2009)

2. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems 25, pp. 1097–1105 (2012)

3. Murai, S., Ushiama, T.: Review-based recommendation of attractive sentences in a novel for effective browsing. Int. J. Knowl. Web Intell. 3(1), 58–69 (Jul 2012)

4. Statista: Number of bookstores in the u.s. 2017. https://www.jpoksmaster.jp/
Info/documents/top_transition.pdf