GROWING MUSEUM
Keyword-based Self-growing CG Virtual Museum with Annotation System
Masaki Hayashi
Uppsala University, Faculty of Arts, Sweden
ABSTRACT
We have been building a keyword-based virtual
museum that allows users to search a database ofartworks by giving search keywords, pick up theartworks, and automatically display them in a CG
museum space of variable dimension. This time, wehave added an annotation system to this museum,allowing users to freely annotate the artworks while
enjoying the exhibition. In this way, the database ofkeyword search is naturally updated, and thedescriptions of the artworks freely viewed in the virtualexhibition is elaborated over time. In this way, while
enjoying the exhibition, the museum itself becomesricher and evolves on its own, which we call theGrowing Museum.
Keywords: keyword search, annotation system, selfgrowing content, virtual reality

1. INTRODUCTION
The importance of virtual museums is increasing. The
virtual museum provides a virtual display of artworks incontrast to a real museum. We have been developing
virtual museums that allow users to freely walkthroughthe realistic museum space using high-quality real-time
computer graphics [1]. The most important requirement
is to create a space where the rooms and artworks of themuseum harmonize well as a whole as if one is enjoying
themself like in a real museum without stress. This time,
we are proposing a system that combines this virtual
museum with an annotation system to update themuseum itself as the user uses it. We named this idea
“Growing Museum”. In this paper, we describe ourapproach, the system design, and the actual test systemthat we are building, using woodblock prints byKatsushika Hokusai.
2. OUR APPROACH
While keeping the aesthetic aspect of a virtual museum
as our basic form, we have been pursuing functions that
only a virtual museum can provide. One of the attempts
is the automatic generation of the exhibition [2][3]. In

this application, a large number of artworks areregistered in a database, and users can search for themby giving keywords to the museum, and the searched
artworks will be automatically displayed in the virtualmuseum. The room of the museum is made to be
expandable and contractable so that the room transforms
according to the number of artworks to be displayed. It
also has a function that allows users to display various
related information about individual works registered in
the database.
The most important thing to consider whenconstructing such a database is how to build it. The moststraightforward way would be to use the informationthat is already bound to each work, mainly that given byprofessional curators. If you access the websites ofmuseums around the world, you will find a lot of suchtext information, and you can use it. In other words,virtual museums have been using this method for a longtime, except for a few experimental attempts [4]. The
famous attempts such as Google Arts & Culture andLouvre Virtual Tours, for example, use the data attachedto the artworks that were made by professionals who
have studied the artworks individually.
We propose a new way to create this data: to providea system that allows art enthusiasts to freely add data to
the artworks. The user can view the work displayed in
the virtual space as if it was the real thing, and annotate
the work on the spot. By repeating this process, theexhibition system itself, including the database, willevolve. We named this approach "Growing Museum" asa concept that the museum itself grows by the hands ofordinary users. This time, we are implementing anexperimental system based on this idea and conducting
an experiment.
3. SYSTEM DESIGN
3.1. Base system - Keyword-based museum
The Growing Museum is built on the keyword-based
virtual museum that we developed. The museum allows
users to give keywords to the system, and the system
will pull in works that match the keywords and display
them in a virtual space. Fig. 1 shows some examples ofan exhibit. As you can see in Fig. 1-b and Fig. 1-c, thewalls of the museum move according to the number of(A) Keyword input (“Moon” in this example)

Fig. 2 Annotation input window

With the keyword-based museum in section 3.1, theuser first enters some keywords to get some artworks
into the virtual space and then walks through it. During
the walk-through, the annotation system in section 3.2
can be activated at any time to update the database ofthe artworks. As a result, the search pattern of themuseum itself and the description of the artworks willchange each time the museum is accessed.
(B) Result of (A), moon-related pictures are displayed(C) Larger room for many pictures
Fig. 1 Screenshots of keyword-based virtual museum

works retrieved according to the keywords, and theexhibition is automatically displayed in the optimal size.
The user can freely walk through this space with themouse to view the artworks.
3.2. Annotation system
Fig. 2 shows the annotation system that the user cangive necessary information to a selected artwork. The
keywords are ranked into 4 levels. Each level has someweight value used by the keyword search of the museum
described in section 3.1. The user can also input/edit thedescription associated with the artwork that will be used
when the user wants to see it while doing walkthrough.
3.3. Growing museum

4. TEST SYSTEM IMPLEMENTATION
We are implementing the system described in chapter 3
on the Unity Game Engine. As a start, we usedapproximately 500 woodblock prints works byKatsushika Hokusai from the Metropolitan Museum(MET)'s Open Access free data. Five hundred images
and their metadata provided by the MET are
downloaded using the MET's open API and are stored
locally. On the other hand, the database of this system isconstructed locally by reading all the original metadata
stored with the images. Initially, the keywords aretemporarily registered with the words in the titles
provided by the MET. Therefore, when the user starts
the museum and enters keywords, this will first hit thetitle of the work and the work will be pulled up. Then,the user is able to modify, add, or delete these keywords
to register them again. This allows that particular user toiterate the process to elaborate the database of
Hokusai’s works. We expect this Hokusai museum
becomes to reflect the user’s personality after theiteration.
5. POTENTIAL OF THE SYSTEM
The more a user uses this system, the more her/his
need is reflected in the exhibition system meaning theartwork database is naturally created and formed.
Therefore, if this artwork database is made into a
package and opened to other people, the people canenjoy experiencing the unique exhibition made by thatperson who could be an amateur Hokusai-lover.
Conversely, if the entire system is deployed on theinternet so that multiple people can access it and rewritethe database of artworks, multiple people cancollaborate to create the exhibition.
Furthermore, if we use a virtual museum with avideo generator, which we developed in the past, users
can view artworks with commentaries added through thesystem, just like watching TV programs [5].
In this way, we aim to open up to the public the wayof interpretation and categorization of artworks, which
until now has still been the work of professional
curators. It could provide a service that allows a wide
range of amateurs to create and publish their own
curating works. By taking this approach, the intention isto change the relationship between artworks, museums,and exhibitions.
REFERENCES[1] M. Hayashi, M. Nakajima, S. Bachelder, A. Iguchi, S.
Machida: “Ultra High Resolution 4K/8K Realtime CG System
and Its Application”, Proceedings of IWAIT2013, Jan. 7-9,(2013)[2] M. Hayashi, S. Bachelder, M. Nakajima. "Automaticgeneration of personal virtual museum." In 2016 International
Conference on Cyberworlds (CW), pp. 219-222. IEEE, 2016.
[3] Self-Museum, https://self-museum.cultural.jp/ (2021, Apr.
22 accessed)[4] Trant, Jennifer, and with the participants in the steve.
museum project. "Exploring the potential for social tagging
and folksonomy in art museums: Proof of concept." New
Review of Hypermedia and Multimedia 12, no. 1 (2006): 83105.
[5] M. Hayashi, S. Bachelder, M. Nakajima. "Virtual Museum
Equipped with Automatic Video Content Generator." ITETransactions on Media Technology and Applications 4, no. 1(2016)