# Storytelling Platform for Virtual Museum Development:
Lifecycle Management of an Exhibition

Chaowanan Khundam, Frédéric Noël

<figure><img src="image_1.png" /></figure>

## To cite this version:

Chaowanan Khundam, Frédéric Noël. Storytelling Platform for Virtual Museum Development: Lifecycle Management of an Exhibition. 14th IFIP International Conference on Product Lifecycle Management (PLM), Jul 2017, Seville, Spain. pp.416-426, 10.1007/978-3-319-72905-3_37. hal-01764205

HAL Id: hal-01764205

https://inria.hal.science/hal-01764205

Submitted on 11 Apr 2018

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
# Storytelling Platform for Virtual Museum
## Development: Lifecycle Management of an Exhibition

Chaowanan Khundam¹ and Frédéric Noël¹

¹ Univ. Grenoble Alpes, CNRS, G-SCOP, 38000 Grenoble, France
{chaowanan.khundam, frederic.noel}@g-scop.inpg.fr

**Abstract.** Digital heritage applications have been widely developed through Virtual Reality (VR) technologies as known as Virtual Museum (VM). Devices and digital contents are significantly increasing and may support interaction system to immerse users into VM. Due to rapid changes of technologies, a platform to develop an exhibition should support to change devices and also to optimize interaction system used in VM. However, usually, both devices organization and contents structure on a platform is still lacking efficient management to support alternative interaction in general. The development and maintenance process of a VM exhibition must be undertaken in an integrated mode. We propose a storytelling platform for developing virtual exhibition with high-level abstractions providing adaptive interaction system. An exhibition's lifecycle management will be useful for maintenance and service in VM when technologies evolved over time. Our framework has 3 sub-processes: development of storytelling platform, development of interaction system and interaction system evaluation. A storytelling template provides a flexible service to manage an exhibition and adapt it to various devices and interaction techniques. Interaction system evaluation will be deployed to maintain an exhibition and support user learning in VM.

**Keywords:** Virtual Reality, Virtual Museum, Digital Storytelling, Interaction system

## 1 Introduction

In digital content era, thousands physical bricks of information are converted to become a digital content. There are many devices and technologies which allow users to access data and to visualize and to interact with virtual world [1]. Virtual reality (VR) technologies have been used for various purposes. Virtual museum is one branch that applies VR for demonstrating actual objects or architecture with mock-up as three-dimensional models [2]. Virtual exhibition applies VR concepts with multimedia information and computer graphic technology. The design and development of virtual museum's exhibition also considers interactivity in the system to allow users to learn about collection of history with content that they are interested in [2]. Virtual museum becomes a large multimedia center where users easily access information and can convey the contents through interaction between users [3]. It allows users to gain knowledge from the museum more easily but the keystone remains a good storytelling. We need a storytelling to drive virtual environment
behavior while the interaction is the modality to follow this story. There is a kind of disconnection between the initial specification of the exhibition and its implementation and delivery. A good product development expects to fill this gap.

Storytelling becomes a major part to create interactive content for a virtual museum and use stories as instruments for suspenseful knowledge transferring [4]. In order to create a virtual museum exhibition, storytelling platforms will be the tool to help developer defining elements in a scene and makes stories. Non-linear storytelling is a kind of interactivity content [5], users get involved into the story to interact with a content. To design and develop an exhibition as non-linear storytelling, the contents will be considered in term of user interaction where interaction system is concerned. Here, devices and interaction techniques are also considered for designing interactive contents. Most of virtual museum's exhibitions have been designed limited to selected devices. Maintenance and service of interaction system will be restricted to the contents of all development process that will be complicated for exhibition management due to the fact that there is no lifecycle management for an exhibition to support technologies change over time.

Therefore, storytelling platform acts as the major specification of any exhibition. We propose high-level abstraction to define all event and action in a scene that must be translated into low-level technical user interactions. Thus, end user (visitor) interactions will respect high-level abstraction whatever devices are finally used. The platform aims to be part of the lifecycle management of an exhibition.

## 2 Related Works

Authoring tools are majority to support an application development. We focus on manual narrative authoring [6] and interaction system is also concerned. Hence, the platform will provide not only story generation but also VR interaction system. Storytelling platform for virtual museum exhibition is related to VR platform which mainly provides functions to develop virtual environments and device plugin support. Educational game-authoring tool is another platform concerned to storytelling with an enjoyment goal. Advantages and disadvantages of each platform will be considered respect to application in the field of virtual museum exhibition.

### 2.1 VR Platforms

There are several platforms for creating a VE by a developer who handle several kinds of devices. VR Juggler [7] is a virtual platform providing a virtual reality environment independent of operating system. This platform supports many graphics engines and network distribution through the NetJuggler module but no high-level support for application distribution. AVANGO/NG [8] is a distributed scene graph framework. It applies a generic field container programming interface based on OpenSceneGraph and develops an entire application with Python scripting support. Vizard [9] is VR toolkit for interactive 3D content, scripting with Python undertaking many commercial devices and also achieving extraordinary rendering including multi-user, clustering, and multi-channel abilities. InVRs [10] implements Collaborative
Virtual Environments (CVEs) approach in the form of a highly extensible, flexible, and modular framework with pre-defined navigation and interaction techniques. Configurable via XML, it has a network distributed virtual world using OpenSG as a scene graph engine. pSIVE [11] platform allows easy setting up of VEs with interactive content and also has a generic model to be applied in different contexts by non-expert users. pSIVE is a good platform to study device switching with pre-defined interaction techniques that selection and navigation techniques will be used to compare potential of each device. 3DVIA Studio [12] is an interactive 3D application platform by LUA scripting language. There are many GUI tools to work on models and animations in scenes. Providing integration with VR/AR systems and also realistic interactive 3D by advance rendering, physics and animation engine support to create immersive virtual reality project. Unity3D [13] is high performance 3D rendering and physics engine for game developer which can be applied for creating VR application. With various device plugin supports, Unity3D is able to export an application towards many operating systems, useful for interaction techniques studying. However, the application development remains based on pre-selected device. To switch device we have to edit programming part for a correct application control due to devices restriction.

Most VR platforms are proposed to be a tool for VEs development and support developer to handle devices configuration. However, storytelling to support and to organize story structure is still lacking. There is some platform providing high-level abstraction to define object behaviors in general for interaction which could be improved as educational game or storytelling platform.

## 2.2 Educational Game-Authoring Tools

WEEV [4] is an educational game creation framework. The system is built upon <e-Adventure> [14], a game authoring platform. WEEV implemented three tools to edit main element which are Actor editor, World editor and Story editor using visual programming language to represent a story based on interaction between user and game. Thinking Worlds [15] is an authoring platform for VEs focused on creating structured learning experiences. It is a commercial tool that facilitates development of serious 3D games. This platform attempts to create more complex scenes which required more 3D rendering engine and also camera and character control through the scene. Adventure Author [16] is a platform based on visual programming language as WEEV is. The development of this platform focuses on linear stories without real interactivity. Thus, it is easy to create educational stories rather than games. StoryTec [17] is another visual programming language like Adventure Author for serious game creation. However, it supports to create non-linear stories to encourage creativity rather than educational game. StoryTec has powerful expressiveness for story creation via a Story editor enabling non-linear stories. Storytelling Alice [18] is proposed to teach programming concept for students but also provides many features to create animated stories. Storytelling Alice includes high-level animation that enables users to program interaction between elements which provides 3D character and scenery with custom animations. Adventure Game Studio [19] and Adventure Maker [20] are platforms for game developer with 2D animation support. In contrast, these platforms
do not provide tools for storytelling or educational features which are not targeted to educator. These platforms have GUI editor to support content editing on the scene based on point and click gaming. If we considered both, Table 1. shows educational game-authoring tools have abilities for developing structured learning and VR platform lacks. By the way, educational game-authoring tools tried to advance their engine to be high performance in 3D while most VR platforms support these features. The creation of an exhibition in virtual museum needs to have a storytelling platform which supports of educational features for user learning but also advanced 3D engine and device switching. Therefore, it would be great to combine good features of each platform together. Furthermore, development process is still lacking of lifecycle management which should be considered into storytelling platform. The process should carry out according to product lifecycle phase as well as conceive, design, realize and service. The design of a storytelling platform for virtual museum exhibition will have methodologies based on exhibition lifecycle management which improves exhibition development process as described in next section.

<table><caption>**Table 1.** Existing environment capacities</caption><thead><tr><th>Platform</th><th>Main approach</th><th>High level</th><th>Storytelling support</th><th>Animation support</th><th>Multi-users</th><th>Device support</th><th>Interaction technique</th><th>Complexity</th></tr></thead><tbody><tr><td>VR Juggler</td><td>XML Scripting</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Many</td><td>Scripting</td><td>High</td></tr><tr><td>AVANGO/NG</td><td>Scripting</td><td>Yes</td><td>No</td><td>No</td><td>Yes</td><td>Many</td><td>NA</td><td>High</td></tr><tr><td>Vizard</td><td>Scripting + GUI Editor</td><td>Yes</td><td>NA</td><td>Yes</td><td>Yes</td><td>Many</td><td>Scripting</td><td>High</td></tr><tr><td>InVRs</td><td>XML Scripting</td><td>NA</td><td>NA</td><td>Yes</td><td>Yes</td><td>Many</td><td>Pre-defined</td><td>High</td></tr><tr><td>3DVIA Studio</td><td>LUA Scripting</td><td>Yes</td><td>No</td><td>Yes</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>High</td></tr><tr><td>pSIVE</td><td>GUI Editor</td><td>No</td><td>No</td><td>NA</td><td>Yes</td><td>Many</td><td>Pre-defined</td><td>Low</td></tr><tr><td>WEEV</td><td>Visual Language</td><td>Yes</td><td>Yes</td><td>Yes, 2D</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>Low/High</td></tr><tr><td>Thinking Worlds</td><td>Hybrid</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>Low</td></tr><tr><td>Adventure Author</td><td>GUI Editor</td><td>NA</td><td>Yes</td><td>NA</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>High</td></tr><tr><td>StoryTec</td><td>Visual Language</td><td>Yes</td><td>Yes</td><td>NA</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>NA</td></tr><tr><td>Storytelling Alice</td><td>Visual Language</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>Low</td></tr><tr><td>Adventure Game Studio</td><td>GUI Editor</td><td>No</td><td>NA</td><td>Yes, 2D</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>High</td></tr><tr><td>Adventure Maker</td><td>GUI Editor</td><td>No</td><td>NA</td><td>No</td><td>No</td><td>Desktop</td><td>Point and Click</td><td>High</td></tr><tr><td>Unity3D</td><td>GUI Editor</td><td>NA</td><td>NA</td><td>Yes</td><td>Yes</td><td>Many</td><td>Scripting</td><td>Very High</td></tr></tbody></table>

## 3 Storytelling Platform

As a top-down design of engineering workflow, storytelling platform is focused on high-level functional requirements. High-level abstraction is decomposed into lower level structures and specifications until the physical implementation layer is reached.
The specification phase is the definition of exhibition requirement. This starts with the design of scene components, defining basic entities and some event controlling and then high-level behaviors will be defined as action logics. From this specification, storytelling platform will be developed. Then the design phase is where the details design and development starts. An exhibition will be developed on the platform as a storytelling project. Next it comes the realization phase; physical implementation is handling here. The complete project will be launched into the low-level functions where interaction system is used. Finally, the service phase serves for interaction system providing system maintenance as well as reuse of project. Collaborative Virtual Environments (CVEs) may support this phase to transfer exhibition project to a specific interaction system. When technology is changed former exhibition project is transferred again for new technology.

<figure><img src="image_3.png" /><figcaption>Fig. 1. Exhibition development architecture</figcaption></figure>

**3.1 Development of Storytelling Platform**

The design of scene components will be applied to develop storytelling platform. The Viewer is a tool to display virtual environment in 3D. All entities are handled in the Asset manager while event and action part will be specified in the Event editor. Here, high-level abstract model will be defined into a scene with events and actions corresponding to storytelling needs. When all entities in a scene have been determined, an execution part will be addressed by a runtime engine to perform interactive stories in real time and interact at low-level connection. The translation between the high-level abstract model and low-level connections should be automated so far as we can do it. Fig. 1. Shows the process and architecture of platform which has the following basic components:

**A Viewer:** A viewer is a display window to present entities as virtual environments and cooperate with Asset manager for entities editing. Viewer provides graphic engine to handle rendering and lighting in the scene. There are 2 modes that use the same window which are storytelling mode for editing and runtime mode to present the result of storytelling mode.

**Asset manager:** Asset manager is a tool panel to carry out all entities imported into the scene. When each model has been imported, entities will be loaded into a manager
panel. Entities can be a representation of a building, artifact, actor, information or avatar. These will keep entities as scene graph data structure.

**Event editor:** event and action have been designed already then implemented to the visual setting in the Storytelling platform. Event editor provides GUI panel to connect each event component together: Entity, Event and Action. Entity will be selected directly in the 3D scene while events and actions are selected in a GUI template. Condition is added into an event for flexibility of logic assignment to be a rule for Action. Event editor is the tool for defining high-level abstractions for every scene, events and actions for all entities provide a common template for every storytelling.

**Runtime engine:** provides runtime mode in the Viewer to test the result from storytelling mode when event and action have been assigned. Event and action are interpreted to execute and perform entities in the scene in real-time interaction. We see how the story is going on and if interaction works well or not. Low-level functions can be implemented and tested on this stage. Before to execute the runtime engine, the connection with low-level interaction system is exported.

**Low-level connection** is a part of a top-down design workflow. High-level abstraction is decomposed into a lower level for physical implementation. This is compatible with vertical transformation as propose in Model Driven approaches [21, 22]. High-level abstraction related to the entity as defined while event and action allow developer to connect the device which is independent of event setting anyway. Here, storytelling platform enables device changing through event and action by Event editor and low-level functions are applied by interaction follows device's characteristic as shown in Fig. 2.

<figure><img src="image_4.png" /><figcaption>Fig. 2. Vertical transformation as model driven approach</figcaption></figure>

## 3.2 Development of Interaction System

Interaction system on our Storytelling platform uses Collaborative Virtual Environments (CVEs) to be the tool for running an application (This tool is internally developed at the laboratory). In order to connect devices to an application, CVE is
expected to manage device connection in the system through a CVE's server. Moreover, interaction techniques (ITs) are methods to let user perform a task within VEs via user interface. ITs are influenced by input devices, but the same device may be used for many ITs for the same task [23]. On the other hand, it may be possible to implement a given IT using several different input devices. ITs studying is essential for system development to use of various devices and to improve system when new technologies arise. Therefore, development of interaction system through CVE will support exhibition lifecycle management especially during maintenance and service phase.

**ITs design:** Interaction tasks are the method to interact with VEs while ITs are the way to perform interaction tasks. To design ITs, the first step will begin with ITs classification. Universal interaction tasks are partitioned into separable subtasks, each of which represents technique component calls taxonomy. Taxonomy is not only a characterization of interaction tasks, but also as a design space. Taxonomy is the intermediate-level of ITs implementation to the lower level. In our Storytelling platform, interaction tasks are separated to be 3 genres of ITs which are Selection, Manipulation and Navigation [21].

**Device setting on platform:** In Event editor, only connected device can be chosen and then ITs will be selected for managing scene behavior. A same input device can select different IT already designed and the same ITs can be used with different input device also. Storytelling platform provides ITs management. Some ITs are probably reuse with other devices by keeping all ITs into a database from where each device can be retrieved.

**Device management:** Usually, CVE is used for collaboration and interaction of multiple users to work together on VR application [24]. However, CVE does not provide only multiple users in the system, but also enable using multiple devices together in the same application. Thus, CVE is the tool which facilitates flexibility when using devices and supports device management of Storytelling platform. Interaction system of our platform will address low-level implementation of device which supports devices service while high-level abstractions maintain the content events and actions in the exhibition.

# 4 Case study

We expect to build an exhibition about history of Prathat temple, the crucial temple model of South East Asia. There have been various legends of Prathat temple. One legend mentioned that the Buddha relics (the teeth of the Lord Buddha) have been kept inside. The relics were moved from Tontha Buri by Prince Thanakuman and Princess Hem-chala. It has been assumed that the original form of Prathat was Mondop, a structure with four arches and a pyramidal roof topped with five tiers as Srivijaya style. Then, in 1700BE, Theravade doctrine of Lanka has been prosperous and spread to Nakhon Si Thammarat, as a result Chedi of Theravade was constructed to cover the original one. This becomes a typical model of Theravade in the south of Thailand and influentially spreads to adjacent areas.
The storyteller expects to make an exhibition to tell this story with Storytelling platform by transforming history in term of text to be an interactive learning exhibition. First, all entities models are loaded into the database as historical model which are able to edit transformation value within the scene. The story is modelled at a high-level abstraction by Event editor which has 3 main components: Entity, Event and Action. Each component is drag and drop node on a canvas. We make relations by linking nodes from Entity to Action through Event which is editable on setting menu. The example scene in Fig. 3 has 4 Entities which are two Actors Prince Thanakuman and Princess Hem-chala and two buildings named Mondop and JDPrathat. At beginning of the scene, we set two actors moving from Marker1 to Marker2 with 5 seconds duration to present the relics were moved from Tontha Buri by them. When approach to Marker2, Mondop will appear to show the original form of Prathat which kept the relics inside and these actors will fade away at the seconds 11. Then the building JDPrathat will be faded-in and we set more input event by adding Selection with respect to device. When this model is selected, the scene will play clicking sound and show message as defined.

Selection event will depend on selected device which can be setting up the interaction technique to select the Building. Here, indication to object is a method to point to any object while indication to select is a selection method. These parameters of IT have been defined respect to device which can choose directly from setting menu. When the device is switched to other one and completely defines new IT. All actions that related to the event still work even device has been changed by flexibility of high-level design. In this example, Selection defined a mouse to be a default device. This platform allows storyteller to change device to other one e.g., a Haptic arm and assign indication to object and to select which calls IT. Although storyteller would like to use the old device, the platform provides capabilities to manage IT directly and still using the same logic. Finally, specification of ITs will be translated into executable interaction system follows selected devices. Fig. 4. Shows high-level abstraction can be transformed the story to various devices.

<figure><img src="image_5.png" /><figcaption>**Fig. 3.** An example of Event editor on drag and drop canvas and selection setting for device and IT management.</figcaption></figure>
<figure><img src="image_6.png" /><figcaption>**Fig. 4.** Transformation of storytelling model to low-level functions with various devices.</figcaption></figure>

**5 Conclusion**

We propose new methodologies for exhibition development in VM to support its management. We believe that it will also support the whole exhibition lifecycle. Devices organization and contents structure are addressed by the Storytelling platform which has efficient management to support alternative interaction in general. Engineering workflow as a top-down design provides a high-level interaction abstraction model where storytelling models specific interaction behavior to drive interaction in a scene. Devices and ITs are handled through a CVE.

According to product lifecycle phase, concept design is supported by our Storytelling platform which specifies the exhibition interaction. Service phase is processed after an evaluation method to keep an exhibition maintained and operational. We claim that Storytelling platform provides an adaptive interaction system which will supports user learning when applied to VM. This implementation is useful to deploy not only for the development of VM exhibition, but also for industrial engineering where interactive content and interaction system are required.

At this step the overall architecture of our exhibition development system is ready. A first demonstrator provides a storytelling modeler and we have a first converter of a story into a runtime environment. We continue developing this platform and we will analyze devices and ITs for each story to classify their specific potential to support user learning in VM.

**Acknowledgments.** This research was supported by Franco-Thai scholarship (French Government) and Walailak University, Thailand.

**References**

1. Styliani S, Fotis L, Kostas K, Petros P (2009) Virtual museums, a survey and some issues for consideration. Journal of cultural Heritage, 10(4), 520-528.
2. Pujol L, (2004) Archaeology, museums and virtual reality. Revista digital de humanidades, UOC. (http://www.uoc.edu/humfil/articles/eng/pujol0304/pujol0304.pdf).

3. Walczak K, Cellary W, White M (2006) Virtual museum exhibitions. IEEE Computer, 39(3), 93-95.

4. Marchiori EJ, Torrente J, Del Blanco Á, Moreno GP, Sancho P, Fernández MB. (2012) A narrative metaphor to facilitate educational game authoring. Computers & Education, 58(1), 590-599.

5. Spaniol M, Klamma R, Sharda N, Jarke M (2006) Web-based learning with non-linear multimedia stories. In International Conference on Web-Based Learning, Springer Berlin Heidelberg, pp. 249-263.

6. Kybartas B, Bidarra R (2016) A survey on story generation techniques for authoring computational narratives. IEEE Transactions on Computational Intelligence and AI in Games.

7. Bierbaum A, Just A, Hartling P, Meinert K, Baker A, Cruz NC (2001) VR Juggler: a virtual platform for virtual reality application development. Proceedings IEEE Virtual Reality 2001, pp. 89-96.

8. Kuck R, Wind R, Riege J, Bogen KM, Birlinghoven S (2008) Improving the avango vr/ar framework: Lessons learned. In Workshop Virtuelle und Erweiterte Realität, pp. 209-220.

9. WorldViz: Vizard VR Software Toolkit. http://www.worldviz.com/products/vizard

10. Anthes C, Volkert J (2006) inVRs-A Framework for Building Interactive Networked Virtual Reality Systems. High Performance Computing and Communications Lecture Notes in Computer Science 4208, pp. 894-904.

11. Souza D, Dias P, Santos D, Santos BS (2014) Platform for setting up interactive virtual environments. In IS&T/SPIE Electronic Imaging, International Society for Optics and Photonics.

12. 3DVIA Studio, Dassault Systemes. http://www.3dvia.com/studio

13. Unity3D game engine. https://unity3d.com

14. Torrente J, Del Blanco Á, Marchiori EJ, Moreno GP, Fernández MB (2010) <e-Adventure>: Introducing educational games in the learning process. In Education Engineering (EDUCON), 2010 IEEE, pp. 1121-1126.

15. Thinking Worlds. http://www.thinkingworlds.com

16. Robertson J, Nicholson K (2007) Adventure Author: a learning environment to support creative design. In Proceedings of the 6th international conference on Interaction design and children, ACM, pp. 37-44.

17. Göbel S, Salvatore L, Konrad R (2008) StoryTec: A digital storytelling platform for the authoring and experiencing of interactive and non-linear stories. In Automated solutions for Cross Media Content and Multi-channel Distribution, AXMEDIS'08, IEEE, pp. 103-110.

18. Kelleher C, Pausch R, Kiesler S (2007) Storytelling Alice motivates middle school girls to learn computer programming. In Proceedings of the SIGCHI conference on Human factors in computing systems, ACM, pp. 1455-1464.

19. Adventure Game Studio. https://www.adventuregamestudio.co.uk

20. Adventure Maker. http://www.adventuremaker.com

21. Calvary G, Coutaz J, Thevenin D, Limbourg Q, Bouillon L, Vanderdonckt J (2003) A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers. Vol. 15, No. 3, pp. 289-308.

22. Coutaz J, Calvary G (2012) Hci and software engineering for user interface plasticity. Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, Human-Computer Interaction, Third Edition, pp. 1195-1220.

23. Bowman DA (1999) Interaction techniques for common tasks in immersive virtual environments. Doctoral dissertation, Georgia Institute of Technology.

24. Wright T, Madey G (2008) A survey of collaborative virtual environment technologies. University of Notre Dame-USA, Tech. Rep, pp. 1-16.