# A Hybrid Approach for Visualizing Spatial and Non-spatial Data Types of Embedded Systems

Ragaad Altarawneh, Shah Rukh Humayoun, Achim Ebert

► To cite this version:

Ragaad Altarawneh, Shah Rukh Humayoun, Achim Ebert. A Hybrid Approach for Visualizing Spatial and Non-spatial Data Types of Embedded Systems. 15th Human-Computer Interaction (INTER-ACT), Sep 2015, Bamberg, Germany. pp.495-498, 10.1007/978-3-319-22723-8_46. hal-01610827

HAL Id: hal-01610827

https://inria.hal.science/hal-01610827

Submitted on 5 Oct 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# A Hybrid Approach for Visualizing Spatial and Non-Spatial Data Types of Embedded Systems

Ragaad AlTarawneh, Shah Rukh Humayoun, Achim Ebert

Computer Graphics and HCI Group
University of Kaiserslautern, 67663 Kaiserslautern, Germany
{tarawneh,humayoun,ebert}@cs.uni-kl.de

**Abstract.** Due to the heterogeneity nature of embedded systems, visualizing them from different perspectives requires including different data sets about the underlying system. This opens the doors to think about integrating between the spatial and the non-spatial data sets in this domain. In this work, we present the SceneMan (Scene Manager) platform that integrates between the different data sets using a 2Dplus3D style to reflect the system from different perspectives. In this context, we visualize the non-spatial data using the normal 2D representation, which can be converted to a 3D representation in some cases such that the stereoscopic depth cue is used to encode some aspects about this data set. Additionally, we visualize the spatial data using 3D visualization techniques. This hybrid solution provides the possibility to bridge the gap between the 2D representations and the 3D visualizations.

**Keywords.** 3D Visualization, Graph Visualization, Stereoscopic Depth, Integrating Spatial Data, Non-Spatial Data.

## 1 Introduction

Many techniques have been proposed to trace the possible failures in embedded systems. The Fault Tree Analysis (FTA) [3] and the Component Fault Tree (CFT) [5] are amongst the most usage techniques. These both mentioned techniques enable safety experts (who are responsible for detecting the failures) to model the possible failure and to trace the failure dependencies between the system components. The resulting fault tree representations model some possible failure scenarios in the system at certain times. However, in order to use such representations system engineers (who are responsible for maintaining all the system components) should be able to understand it and map it with the actual system structures. The collaboration between safety experts and system engineers is important in order to analyze the system behavior and to eliminate the possible critical situations during the system life [1]. However, this collaboration faces several challenges, e.g., the divergence between the interested perspectives. Further, there is a variation in data representations for both groups, e.g., the hardware model of the underlying system (normally used by system engineers) is mostly represented as a 3D data type while the safety scenario (normally used by
safety experts) is mostly modeled as a fault tree and is considered to be an abstract data type, which can easily be represented as a 2D graph representation.

In order to fill the above-mentioned gap, we propose a hybrid solution, called the **SceneMan** (*Scene Manager*) platform, to visualize the two data types in a 2Dplus3D world, such that the 2D graph representation is available to visualize the abstract fault tree structure while the 3D part is used to visualize the system hardware model. Further, we propose to visualize this 2Dplus3D world using the stereoscopic displays such that the stereoscopic depth is utilized to encode some aspects about the nodes (e.g., to encode the structural relations) in the graph representation. In this case, the depth is used together with other visual cues to encode or convey some data aspects.

## 2 The SceneMan Platform

In Figure 1, we show the overall hybrid approach behind the SceneMan platform. SceneMan is responsible for integrating the system 3D view with the graph representation of the abstract data. For this, it combines and synchronizes the system 3D view together with the abstract graph 2D view for creating a 2Dplus3D environment. It supports these views' alignment in order to respect each other in the 3D world (see Figure 2). It also gives the viewers the ability to control the positions of each view for getting more insight about the underlying data.

<figure><img src="image_2.png" /><figcaption>**Fig. 1.** The mapping operation in the SceneMan hybrid approach</figcaption></figure>

SceneMan offers different interaction techniques for interacting with each view individually without distorting the other view. This is achieved through the *scene-graph* concept. In this case, the root node represents the whole 3D space containing all the
views while children of this root are the 2D abstract view using the graph representa-
tion of the component fault tree and the 3D model view of the system.

In order to minimize the occlusion effects, elements in the 3D space are arranged
together especially when the size of any of the two views is changed. For this, the size
of non-focused view is adapted consequently to guarantee a readability increment of
the current focused view. The initial *side-by-side* configuration is changed when the
viewer starts interacting with the 3D space. For example, both views can be arranged
into a *layered* view where these views are stacked above each other, as shown in Fig-
ure 2. In this case, the depth cue is used to show the importance degree of the view
Moreover, viewers can also get more focus on the top-view by increasing the distance
in depth between it and the lower view. Additionally, SceneMan provides the facility
to adjust the size of the view based on the degree of interest in the required view.

<figure><img src="image_3.png" /><figcaption>**Fig. 2.** An integrated *Layered* view showing the correlation between the two views (i.e., the 2D abstract view and the system 3D model view) using the stack metaphor.</figcaption></figure>

3 Utilizing the Stereoscopic Depth

SceneMan visualizes the safety aspects of embedded systems using the compound graph representation, where these graphs represent the fault tree models to describe the possible critical situations. Because these models represent abstract non-spatial data; therefore, SceneMan visualizes them using 2D representations, as it is commonly accepted visual representation approach for this data type. However, in order to integrate it with the 3D environment of the platform, we designed it in a way that bridges the representation gap between the two data types. This is achieved by rendering the overall system in a stereoscopic platform, where the stereoscopic depth is used to encode some aspects about the nodes in the graph representation.

Based on our knowledge, this is the first approach that integrates between the 2D
representations and the 3D technologies through encoding structural relations in com-
pound graphs using the stereoscopic depth cue. SceneMan visualizes compound
graphs using multi-level graph layout algorithms. In this context, it provides different
levels of details inside the graph structure. For example, in Figure 3 all the nodes are
clustered in the initial view; however, they can be explored using the *expanding* and
the *contracting* operations of the CluE algorithm [2] used inside the SceneMan. Once
any compound node is expanded, a new level of details is shown up containing the
expanded node's internal structure. This level is visualized using a new plane that is
created closer to the viewer than the remaining graph parts that are not explored yet.
The children of the expanded node are positioned in a depth value closer to the viewer
than the other graph parts. This approach produces a layered graph representation.
However, due to the stereoscopic effect these elements look sharper than the remain-
ing graph parts, which gives a natural *focus+context* interaction technique, i.e., the
closer elements are in focus while the unexplored parts are in the background of the
scene to keep the context of the whole graph structure.

<figure><img src="image_4.png" /><figcaption>**Fig. 3.** *a)* The initial compound graph, *b)* the front view after expansion of two nested compound nodes; and *c)* the side view of (b).</figcaption></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

Additionally, SceneMan allows changing interactively other graph settings, e.g., the transparency of the depth layers in which the further layers look more transparent than the closer layers in the 3D world. The transparency effect was added as it was proposed by Eades and Feng [4] to show the hierarchical relations of clustered graphs. This parameter is useful in decreasing the occlusion ratio between the different visual objects at different depth layers. Naturally, this increases the emphasis to the closer objects reside in the closer layers.

In the future, we intend to evaluate our system with users from different back-
ground as well as to evaluate it in other 3D platforms like CAVE systems or power-
wall screens.

4 References

1. AlTarawneh, R., Bauer, J., Humayoun, S. R., Ebert, A., and Liggesmeyer, P. Enhancing Understanding of Safety Aspects in Embedded Systems Through an Interactive Visual Tool. *IUI Companion '14*, ACM, New York, NY, USA, 9-12, 2014.

2. AlTarawneh, R., Schultz, J., Humayoun, S. R. CluE: An Algorithm for Expanding Clustered Graphs. *PacificVis 2014*, IEEE, 233-237, 2014.

3. M. Bozzano and A. Villafiorita. Design and Safety Assessment of Critical Systems. *CRC Press (Taylor and Francis)*, an Auerbach Book, 2010.

4. P. Eades and Q.-W. Feng. Multilevel Visualization of ClusteredGraphs. *In Proc. Graph Drawing*, GD, number 1190, pages 101–112, 1996.

5. B. Kaiser, P. Liggesmeyer, and O. Mckel. A New Component Concept for Fault Trees. *Reproduction*, 33:37–46, 2003.