Remote Visualization Tool PBVR(v1.04) User
Guide

March 2015
Japan Atomic Energy Agency
Center for Computational Science & e-Systems

Revision Record
Version

Revised

Revisednumberdate

chapter

1.04-1

2015.3.31-

Revised content
Releasei

Contents
１ Introduction ..................................................................................................................................... 4
１.１. Overview .................................................................................................................................. 4
１.２. System Environment ............................................................................................................. 5
２ Installation........................................................................................................................................ 6
２.１. Filter .......................................................................................................................................... 6
２.１.１. Installation of Load Module ......................................................................................... 6
２.１.２. Installation from Source Code..................................................................................... 7
２.２. Server ....................................................................................................................................... 7
２.２.１. Installation of Load Module ......................................................................................... 7
２.２.２. Installation from Source Code..................................................................................... 8
２.２.３. Windows........................................................................................................................... 9２.３. Client ......................................................................................................................................... 9
２.３.１. Installation of Load Module ....................................................................................... 10
２.３.２. Installation from Source Code................................................................................... 10
２.３.３. Linux, Mac...................................................................................................................... 10
２.３.４. Windows......................................................................................................................... 10
３ Filter ................................................................................................................................................ 12３.１. Overview ................................................................................................................................ 12３.２. Data decomposition model ................................................................................................ 12３.３. Launch Filter ......................................................................................................................... 15
３.４. File Format ............................................................................................................................. 15
３.４.１. Input Data Format ....................................................................................................... 16
３.４.２. Endian ............................................................................................................................ 16
３.４.３. Filter Output Information File (.pfi) ........................................................................... 17
３.４.４. SPLIT File Format ......................................................................................................... 19
３.４.５. Sub-volume Aggregate Format ............................................................................... 22
３.４.６. Step Aggregate Format .............................................................................................. 25
３.５. Parameter File ....................................................................................................................... 29
３.６. MPI Parallel Processing ....................................................................................................... 30
３.７. Execution Method in Staging Environment.................................................................... 31
３.７.１. Execution Shell Script and Parameter File ............................................................. 32
３.７.２. Input-output Files and Directories ........................................................................... 33
３.８. Unstructured Grid data with Mixed Elements ............................................................... 34
４ Server ............................................................................................................................................. 36ii

４.１. Overview ................................................................................................................................ 36
４.２. Launch Server ...................................................................................................................... 36
４.２.１. Launch in Batch Mode ............................................................................................... 37
４.２.２. Launch in Client-Server Mode .................................................................................. 38
４.２.３. Connecting Client and Server by Socket Communication .................................. 38
５ Client ............................................................................................................................................... 44
５.１. Overview ................................................................................................................................ 44
５.２. Launching Client .................................................................................................................. 44
５.３. Terminating PBVR ............................................................................................................... 46
５.３.１. Standard Termination ................................................................................................. 46
５.３.２. Forced Termination ..................................................................................................... 46
５.４. Operating Instructions ........................................................................................................ 47
５.４.１. Viewer............................................................................................................................. 47
５.４.２. Main Panel ..................................................................................................................... 48
５.４.３. CROP Panel ................................................................................................................... 50
５.４.４. Transfer Function Editor ............................................................................................ 52
５.４.５. Time panel ..................................................................................................................... 62
５.４.６. Animation panel ........................................................................................................... 63
６ Examples Using Sample Data ................................................................................................... 64
６.１. Filter Processing ................................................................................................................... 64
６.２. Starting PBVR ...................................................................................................................... 65
６.３. Designing Transfer Function ............................................................................................ 67
６.３.１. Volume Rendering of Single Variable ..................................................................... 67
６.３.２. Multivariate Volume Rendering ................................................................................ 68
６.３.３. Slice Plane ..................................................................................................................... 68
６.３.４. Synthesis of Transfer Functions .............................................................................. 70
６.４. Preserving Result ................................................................................................................. 71
６.５. Example of Batch Mode ..................................................................................................... 72

iii

１ Introduction
１.１. Overview
This manual describes a remote visualization system PBVR, developed at the Center for
Computational Science & e-Systems, Japan Atomic Energy Agency. The system realizes
high-speed remote visualization of large-scale volume data using the KVS library and theParticle Based Volume Rendering (PBVR) technique developed at the Koyamada
Visualization Laboratory, Kyoto University. The system consists of the following threecomponents.
（１） Filter
Filter reads the original volume data and divides it into sub-volumes which areprocessing units for parallel visualization.
（２） PBVR Server
PBVR Server reads sub-volumes for the parallel visualization, and generates
particle data based on the PBVR technique.
（３） PBVR Client
PBVR Client receives the particle data, and renders it to display images using OpenGL.

Figure 1 The whole configuration of the visualization application system

4

１.２. System Environment
The system has been verified on the following environments.
●Filter
Platform

Compiler

Linux 64bit

gcc version 4.4.6

Mac 64bit※2

gcc version 4.8.2

Windows7 64bit

Visual Studio 2013

K Computer

Fujitsu Compiler

FX10

Fujitsu Compiler

BX900

Fujitsu Compiler

Library●Server
Platform

Compiler

LibraryLinux 64bit

g++ version 4.4.6

kmath, ntl※1

Mac 64bit※2

g++ version 4.8.2

Windows7 64bit

Visual Studio 2013

K Computer

Fujitsu Compiler

kmath, ntl※1

FX10

Fujitsu Compiler

kmath, ntl※1

BX900

Fujitsu Compiler

kmath, ntl※1

Platform

Compiler

LibraryLinux 64bit

g++ version 4.4.6

OpenGL, GLUT

Mac 64bit※2

g++ version 4.8.2

OpenGL, GLUT

Windows7 64bit

Visual Studio 2013

OpenGL, GLUT

●Client※1. kmath is a massively parallel random number library developed at the RIKEN,
Advanced Institute for Computational Science. kmath uses the ntl library.
※2. On Mac, OpenMP is not available on the default version of gcc installed with Xcode.
To use load modules or to compile source codes, newer verions of gcc compilers with
OpenMP are needed. This binary was compiled with gcc48 installed throughMacports. gcc can be installed through Macports as follows.
$port install gcc48$port select –set gcc mp-gcc48

5

２ Installation
PBVR is provided as a load module package, load_module_v1.04.tgz, and a source code
package, pbvr_v1.04.tgz. The following sections show how to install Filter, Server, andClient.

２.１. Filter
The filter program is implemented in C, and a MPI+OpenMP version for massively parallel
computing and an OpenMP version for thread parallel computing are available.

２.１.１. Installation of Load Module
The following table shows a list of load modules stored in the filter directory of the load
module package. To complete the installation, copy the load modules in each environment
to an appropriate directory where PATH is set up.
Table 1

List of load modules for the filter program

Platform

Parallelization

Name of load module

Linux 64bit

OpenMPfilter_Linux_omp

MPI+OpenMP

filter_Linux_mpi_omp

Mac 64bit

OpenMPfilter_mac

Windows7 64bit

OpenMPfilter_Win_omp.exe

K Computer※1

OpenMPfilter_K_omp

MPI+OpenMP

filter_K_mpi_omp

OpenMPfilter_FX10_omp

MPI+OpenMP

filter_FX10_mpi_omp

OpenMPfilter_BX_omp

MPI+OpenMP

filter_BX_mpi_omp

FX10※1

BX900※1※1. The load modules for supercomputers are used only on computing nodes. The load
modules for Linux should be used on login nodes and post-processing nodesconsisting of Linux servers.

6

２.１.２. Installation from Source Code
Extract a source code package, filter.tgz, to an arbitrary directory and compile it using
Makefile listed below. To complete the installation, copy generated load modules to an
appropriate directory where PATH is set up.
Table 2

List of Makefiles for the filter program

Platform

Parallelization

Name of Makefile

Linux 64bit

OpenMP

Makefile.linux_omp

MPI+OpenMP

Makefile.linux_mpi_omp

Mac 64bit

OpenMP

Makefile.mac

Windows7 64bit※1

OpenMP

Makefile.win

K Computer※2

OpenMP

Makefile.omp

MPI+OpenMP

Makefile.mpi_omp

OpenMP

Makefile.fx10_omp

MPI+OpenMP

Makefile.fx10_mpi_omp

OpenMP

Makefile.bx_omp

MPI+OpenMP

Makefile.bx_mpi_omp

FX10※2

BX900※2※1. On the Windows environment, launch the VS2013 x64 Native Tools commandprompt to use nmake.
※2. Makefiles for supercomputers are designed for cross-compilers on login nodes.

２.２. Server
The server program is implemented in C++, and a MPI+OpenMP version for massively
parallel computing and an OpenMP version for thread parallel computing are available.

２.２.１. Installation of Load Module
The following table shows a list of load modules stored in the sever directory of the load
module package. To complete the installation, copy the load modules in each environment
to an appropriate directory where PATH is set up.
Table 3

List of load modules for the server program

Platform

Parallelization

Name of load module

Linux 64bit

OpenMP

CPUServer_linux_omp

MPI+OpenMP

CPUServer_linux_mpi_omp

7

Mac 64bit

OpenMP

CPUServer_mac

Windows7 64bit

OpenMPPBVR_Server_win.exe

K Computer※1

OpenMP

CPUServer_k_omp

MPI+OpenMP

CPUServer_k_mpi_omp

OpenMP

CPUServer_fx10_omp

MPI+OpenMP

CPUServer_fx10_mpi_omp

OpenMP

CPUServer_bx_omp

MPI+OpenMP

CPUServer_bx_mpi_omp

FX10※1

BX900※1※1. The load modules for supercomputers are used only on computing nodes. Use theload modules for Linux on login nodes and post-processing nodes consisting of aLinux server.

２.２.２. Installation from Source Code
Extract a source code package, server.tgz, to an arbitrary directory and compile it using
Makefile listed below. To complete the installation, copy generated load modules to an
appropriate directory where PATH is set up.
Table 4

List of Makefiles for the server program

Platform

Parallelization

Name of Makefile

Linux 64bit※1

OpenMP

makefile.linux_omp

MPI+OpenMP

makefile.linux_mpi_omp

Mac 64bit※3

OpenMP

makefile.mac

K Computer※2

OpenMP

makefile.k_omp

MPI+OpenMP

makefile.k_mpi_omp

OpenMP

makefile.fx10_omp

MPI+OpenMP

makefile.fx10_mpi_omp

OpenMP

makefile.bx_omp

MPI+OpenMP

makefile.bx_mpi_omp

FX10※2

BX900※2※1. kmath provided in the Linux environment is compiled using OpenMPI. To use other
MPI libraries, recompile kmath as follows.
[Step１]
Modify the following file depending on the system environment.
./server/kmath_linux/random/Makefile.machine[Step２]$cd ./server/kmath_linux/random/c++
$make

8

When the compilation of numerical libraries, ntl and kmath, fails, modify the following
lines in Makefile in order to compile the server program without using the numericallibraries.
Delete ‘–DKMATH’ from ‘CXXFLAGS=’
Delete ‘-lkm_random –lntl’from ‘LDFLAGS=’
Note that, in this case, the independence of pseudo random numbers is notguaranteed in the MPI+OpenMP version. Therefore, the use of kmath is recommended
on the MPI+OpenMP version.

※2. Makefiles for supercomputers are designed for cross-compilers on login nodes.
※3. The Mac version does not use numerical libraries, ntl and kmath.

２.２.３. Windows
The Windows version does not use the numerical libraries, ntl and kmath, and it iscompiled as follows.
（１） Extract the server environment on a machine with the Visual Studio 2013environment.
（２） On the serverdirectory, run ‘\\project_vc\PBVR_Server_win.sln’ and launch
Visual Studio 2013.
（３） Choose Release x64 as a configuration (see Figure 2).

Figure 2

Sample configuration of Visual Studio 2013

（４） Go to Visual Studio 2013 Menu [Build] ⇒ Run [Build solution].
（５） A

load

module,PBVR_Server_win.exe,‘\\project_vc\x64\Release’.

２.３. Client
The client program is implemented in C++ and OpenGL.

9iscreatedunder２.３.１. Installation of Load Module
The following table shows a list of load modules stored in the client directory of the load
module package. To complete the installation, copy the load modules in each environment
to an appropriate directory where PATH is set up.
Table 5

List of load modules for the client program

Platform

Parallelization

Name of load module

Linux 64bit

OpenMPPBVRViewer_linux

Mac 64bit

OpenMPPBVRViewer_mac

Windows7 64bit※1

OpenMPPBVRViewer_win.exe※1. For the Windows version, copy ‘glut32.dll’ to the same directory.

２.３.２. Installation from Source Code
Extract client.tgz to an arbitrary directory and compile it using a compilation script. To
complete the installation, copy generated load modules to an appropriate directory where
PATH is set up.

２.３.３. Linux, Mac
Execute a compilation script as follows.
$ cd ./client/PBVRViewer$ sh build_cpu.sh※1. To compile all the sources without using past object files, use ‘rebuild_cpu.sh’.

２.３.４. Windows（１） Installation of glut（64bit）①

Download ‘glut-3.7.6-bin_x64.zip(64bit)’ from the link below.
http://ktm11.eng.shizuoka.ac.jp/lesson/modeling.html②

Extract ①.
glut.hglut32.lib
glut32.dll（２） Compilation of glui-2.36①

Go to ‘glui-2.36\src\msvc’.

②Open ‘glui.sln’ in VisualSutudio 2013.
10

③

Choose Release x64 for a configuration as Figure 2.

④

Go to Solution Explore and right click on ‘glui’, open property [configuration
property] →[C/C++], and add a path of ‘glut.h’ to the additional include
directory.

⑤

Go to Solution Explore and right click on ‘glui’, and execute build.

（３） Compilation of ‘KVS-src-1.1.1’①

Go to ‘KVS-src-1.1.1’ and copy ‘build_win.bat’ with a different name.

②

Change ‘kvs.conf’ as follows.
KVS_SUPPORT_GLEW

③

=1

⇒

KVS_SUPPORT_GLEW=0Specify ‘[Directry storing PBVRVlewer.sln]\kvs’ for ‘KVS_DIR’ in the bat
file in ①.

④

Specify ‘[Directory storing glut.lib]’ for ‘KVS_GLUT_DIR’ in the bat file in①.

⑤

Go to [start menu]→[Visual Studio 2013]→[Visual Studio Tool] and open
“VS2013 x64 Native Tools command prompt.”⑥

Go to ‘KVS-src-1.1.1’ and execute the bat file created in ①③④.

（４） Compilation of PBVR Client①

Choose Release x64 for a configuration.

②

Go to Solution Explore and right click on PBVRViewer, open property[configuration property] →[C/C++], and add an include path of ‘glut’ to“additional include directory.”③

Go to Solution Explore and right click on ‘PBVRViewer’, open property[configuration property] → [linker], and add a ‘lib’ path of ‘glut’ to“additional library directory.”④

Go to [menu] → [solution build] to execute the compilation⑤

A load module, PBVRViewer.exe, is created under ‘\\x64\Release\’.

⑥

When execute ⑤, place ‘glut32.dll’ in the same directory.

11

３ Filter３.１. Overview
The filter program is an independent preprocessing program from the PBVR system. The
program divides time-series volume data for parallel processing of a PBVR server, andgenerates sub-volume data for visualization. The data decomposition is based on theoctree model, and the program divides structured grids data and unstructured grids data
into arbitrary octree regions to generate input files for parallel processing of the PBVR
Server.

３.２. Data decomposition model
As shown in Figure 3, the octree structure is generated by dividing each side of a cuboidin half and generating eight sub-cuboids. By repeating the division procedure recursively,
the volume is segmented. Each cuboid has eight child cuboids while child cuboids have asingle parent cuboid.

12

Figure 3 Domain decomposition with the octree structure

As shown in Figure 4, the boundaries of the child-cuboids can be computed by dividing
the sum of the minimum and maximum values in each coordinate by two. The inclusion
relation of each vertex can be determined by comparing the coordinates of the vertex andthe boundaries.

13

Figure 4 Coordinate values of the boundaries in the octree structure14

３.３. Launch Filter
By specifying a parameter file name in the command line option, the filter program is
executed with parameters described in the parameter file. When no parameter file name ora non-existent file name is specified, the execution fails.
（launch in MPI+OpenMP processing with N processes）$ mpiexec –n N ./filter param.txt（launch only in OpenMP processing）$./filter param.txt※1. In both processing modes, the number of OpenMP threads is set in the environmental
variable‘OMP_NUM_THREADS’.
※2. In Windows, the command is launched from VS2013 x64 Native Tools commandprompt.

３.４. File Format
This section describes file formats used in the filter program. All binary format data in the
output files is given in single precision, no header/footer, and little endian. Three fileformats are available including SPLIT format (KVSML format), sub-volume aggregate
format, and step aggregate format (see Figure 5). The SPLIT format generates
independent files for each step and sub-volume. However, in this format, the number offiles grows explosively as the layer of octree. To resolve this problem, the sub-volume
aggregate format, which aggregates files at different time step (and at the same
sub-volume) in a single file, and the step aggregate format, which aggregates files atdifferent sub-volume (and at the same time step) in a single file, are proposed. The
following sections describe their file formats in details.

15

Figure 5

Three file formats

３.４.１. Input Data Format
The filter program can process the following input data.
（１）

AVSFLD binary data

（２）

AVSUCD binary data※1，２※1.

AVSUCD binary data with ‘data’ format can be used. ‘geom format’ and‘data_geom format’ are not supported.

※2.

2D and 3D elements in Table 8 and their mixed elements are supported.

For more details of AVS data formats, refer to the following link (Japanese) or the AVS
manual.
http://www.cybernet.co.jp/avs/products/avsexpress/dataformat.html

３.４.２. Endian
Binary files used in the filter program are unified as little endian. On a machine employing
big edian, visualization data needs to be produced in little edian or to be converted to little
edian in advance before executing the filter program.

16

３.４.３. Filter Output Information File (.pfi)
This file contains binary data summarizing the information about the volume data.
Total number of node(int)
Total number of element(int)Element type(int)※1
File type(int)※2
Number of file(int)※3
Number of component(int)
Start step(int)
End step(int)
Number of sub-volume(int)※4
Whole 3 dimensional space X-axis minimum value(float)
Whole 3 dimensional space Y-axis minimum value(float)
Whole 3 dimensional space Z-axis minimum value(float)
Whole 3 dimensional space X-axis maximum value(float)
Whole 3 dimensional space Y-axis maximum value(float)
Whole 3 dimensional space Z-axis maximum value(float)
Sub-volume 1 node number(int)Sub-volume 2 node number(int)Sub-volume 3 node number(int)：Sub-volume n node number(int)Sub-volume 1 element number(int)Sub-volume 2 element number(int)Sub-volume 3 element number(int)：Sub-volume n element number(int)Sub-volume 1 X-axis minimum value(float)
Sub-volume 1 Y-axis minimum value(float)
Sub-volume 1 Z-axis minimum value(float)
Sub-volume 1 X-axis maximum value(float)
Sub-volume 1 Y-axis maximum value(float)
Sub-volume 1 Z-axis maximum value(float)
Sub-volume 2 X-axis minimum value(float)
Sub-volume 2 Y-axis minimum value(float)
Sub-volume 2 Z-axis minimum value(float)
Sub-volume 2 X-axis maximum value(float)
Sub-volume 2 Y-axis maximum value(float)
Sub-volume 2 Z-axis maximum value(float)：
Sub-volume n X-axis minimum value(float)
Sub-volume n Y-axis minimum value(float)
Sub-volume n Z-axis minimum value(float)
Sub-volume n X-axis maximum value(float)
Sub-volume n Y-axis maximum value(float)
Sub-volume n Z-axis maximum value(float)
17

Step 1 variable 1 minimum value
Step 1 variable 1 maximum value
Step 1 variable 2 minimum value
Step 1 variable 2 maximum value：
Step 1 variable N minimum value
Step 1 variable N maximum value：
Step m variable 1 minimum value
Step m variable 1 maximum value
Step m variable 2 minimum value
Step m variable 2 maximum value：
Step m variable N minimum value
Step m variable N maximum value※1. Refer to Table 8 for the definition of elemeyt types.
※2. The following three formats are available.
・0：SPLIT format

・1：Sub-volume aggregate format

・2：Step aggregate format※3. When a file type is the sub-volume aggregate format, specify the total numberof files.
※4. The number of sub-volumes is expressed as ‘8n_layer’.
・n_layer=0：1・n_layer=1：8・n_layer=2：64・n_layer=3：512・n_layer=4：4,096・n_layer=5：32,768・n_layer=6：262,144・n_layer=7：2,097,152

18

３.４.４. SPLIT File Format
An element configuration file and a node coordinate file are produced for each
sub-volume. A ‘kvsml’ file and a variable file are generated for each sub-volume ans
step. Accordingly, the total number of files is caluculated from the following formula.
The number of sub-volume×2 + the number of sub-volume×the number of step×2(Example: n_layer=7&step=100：423,624,704)

３.４.４.１ File Name
prefix_XXXXX_YYYYYYY_ZZZZZZZ.kvsml………kvsml file（ASCII format）

prefix_YYYYYYY_ZZZZZZZ_connect.dat………element configuration file（binary

format）
prefix_YYYYYYY_ZZZZZZZ_coord.dat… … … node coordinate file （ binary

format）
prefix_XXXXX_YYYYYYY_ZZZZZZZ_value.dat………variable file（binary format）

XXXXX：Number of steps（5 digit number）

YYYYYYY：Sub-volume number（7 digit number）

ZZZZZZZ：Total number of Sub-volumes（7 digit number）

３.４.４.２ Kvsml File Format<?xml version="1.0" ?><KVSML><Object type="UnstructuredVolumeObject"><UnstructuredVolumeObject cell_type=" type of elements"><Node nnodes="number of nodes in the sub-volume"><Value veclen="number of variables"><DataArray type="float" file="prefix_XXXXX_YYYYYYY_ZZZZZZZ_value.dat" format="binary" /></Value><Coord><DataArray type="float" file=" prefix_YYYYYYY_ZZZZZZZ_coord.dat" format="binary" /></Coord></Node><Cell ncells="number of elements in the sub-volume"><Connection><DataArray type="uint"

file=" prefix_YYYYYYY_ZZZZZZZ_connect.dat" format="binary" /></Connection></Cell></UnstructuredVolumeObject></Object></KVSML>19

３.４.４.３ Element Configuration File Format
Element 1 configuration node 1
Element 1 configuration node 2：
Element 1 configuration node nElement 2 configuration node 1
Element 2 configuration node 2：
Element 2 configuration node nElement 3 configuration node 1
Element 3 configuration node 2：
Element 3 configuration node n：Element N configuration node 1
Element N configuration node 2：
Element N configuration node n

３.４.４.４ Node Coordinate File Format
Node 1
Node 1
Node 1
Node 2
Node 2
Node 2
Node 3
Node 3
Node 3

X coordinate
Y coordinate
Z coordinate
X coordinate
Y coordinate
Z coordinate
X coordinate
Y coordinate
Z coordinate：：
Node m X coordinate
Node m Y coordinate
Node m Z coordinate

20

３.４.４.５ Variable File
Node 1 Variable 1
Node 2 Variable 1
Node 3 Variable 1：
Node n Variable 1
Node 1 Variable 2
Node 2 Variable 2
Node 3 Variable 2：
Node n Variable 2
Node 1 Variable mNode 2 Variable mNode 3 Variable m：Node n Variable m

21

３.４.５. Sub-volume Aggregate Format
Element configuration, node coordinates, and variables of all time steps are divided into
sub-volumes. By specifying ‘Number of file’, one can aggregate the information ofseveral sub-volumes into arbitorary number of files from one to the number ofsub-volumes. （n_layer=7：2,097,152）

３.４.５.１ File Name
prefix_YYYYYYY_ZZZZZZZ.dat ………（Binary format）
prefix：Prefix

YYYYYYY：File number（7 digit number）

ZZZZZZZ：Total number of files（7 digit number ）22

３.４.５.２ File Format
Start sub-volume number
End sub-volume number
Element 1 configuration node 1
Element 1 configuration node 2：
Element 1 configuration node n：Element N configuration node 1
Element N configuration node 2：
Element N configuration node n
Node 1 X coordinate
Node 1 Y coordinate
Node 1 Z coordinate：
Node m X coordinate
Node m Y coordinate
Node m Z coordinate
：
：
：
：
：
：
：
Element 1 configuration node 1
Element 1 configuration node 2：
Element 1 configuration node n：Element N configuration node 1
Element N configuration node 2：
Element N configuration node n
Node 1 X coordinate
Node 1 Y coordinate
Node 1 Z coordinate：
Node m X coordinate
Node m Y coordinate
Node m Z coordinate
Node m X coordinate
Node m Y coordinate
Node m Z coordinate

Sub-volume information

Element
configuration

Sub-volume 1

Nodecoordinate

Element
configuration

Sub-volume nNodecoordinate

23

Step 1 node 1 variable 1
Step 1 node 2 variable 1：
Step 1 node n variable 1：
Step 1 node 1 variable m
Step 1 node 2 variable m：
Step 1 node n variable m：：
Step 1 node 1 variable 1
Step 1 node 2 variable 1：
Step 1 node n variable 1：
Step 1 node 1 variable m
Step 1 node 2 variable m：
Step 1 node n variable m：：：：
Step N node 1 variable 1
Step N node 2 variable 1：
Step N node n variable 1：
Step N node 1 variable m
Step N node 2 variable m：
Step N node n variable m：：
Step N node 1 variable 1
Step N node 2 variable 1：
Step N node n variable 1：
Step N node 1 variable m
Step N node 2 variable m：
Step N node n variable m

Sub-volume unit

Step unit

Sub-volume unit

Sub-volume unit

Step unit

Sub-volume unit24

３.４.６. Step Aggregate Format
This format has an element configuration file and a node coordinate file, containing all
the sub-volume information. A variable file is produced for each step. The total number offiles becomes the number of steps + 2.

３.４.６.１ File Name
prefix_connect.dat………Element configuration file（binary format）

prefix_coord.dat………Node coordinate file（binary format）

prefix_XXXXX_value.dat ………Variable file（binary format）
prefix：Prefix

XXXXX ：Number of steps（5 digit number）

25

３.４.６.２ Element Configuration File Format
Sub-volume 1 element 1 configuration node 1
Sub-volume 1 element 1 configuration node 2：
Sub-volume 1 element 1 configuration node n
Sub-volume 1 element 2 configuration node 1
Sub-volume 1 element 2 configuration node 2：
Sub-volume 1 element 2 configuration node n
Sub-volume 1 element 3 configuration node 1
Sub-volume 1 element 3 configuration node 2：
Sub-volume 1 element 3 configuration node n：
Sub-volume 1 element N configuration node 1
Sub-volume 1 element N configuration node 2：
Sub-volume 1 element N configuration node n：
Sub-volume M element 1 configuration node 1
Sub-volume M element 1 configuration node 2：
Sub-volume M element 1 configuration node n
Sub-volume M element 2 configuration node 1
Sub-volume M element 2 configuration node 2：
Sub-volume M element 2 configuration node n
Sub-volume M element 3 configuration node 1
Sub-volume M element 3 configuration node 2：
Sub-volume M element 3 configuration node n：
Sub-volume M element N configuration node 1
Sub-volume M element N configuration node 2：
Sub-volume M element N configuration node n26

Element
unit

Sub-volumeunit３.４.６.３ Node Coordinate File Format
Sub-volume 1 node 1 X coordinate
Sub-volume 1 node 1 Y coordinate
Sub-volume 1 node 1 Z coordinate
Sub-volume 1 node 2 X coordinate
Sub-volume 1 node 2 Y coordinate
Sub-volume 1 node 2 Z coordinate
Sub-volume 1 node 3 X coordinate
Sub-volume 1 node 3 Y coordinate
Sub-volume 1 node 3 Z coordinate：：
Sub-volume 1 node m X coordinate
Sub-volume 1 node m Y coordinate
Sub-volume 1 node m Z coordinate：：：
Sub-volume M node 1 X coordinate
Sub-volume M node 1 Y coordinate
Sub-volume M node 1 Z coordinate
Sub-volume M node 2 X coordinate
Sub-volume M node 2 Y coordinate
Sub-volume M node 2 Z coordinate
Sub-volume M node 3 X coordinate
Sub-volume M node 3 Y coordinate
Sub-volume M node 3 Z coordinate：：
Sub-volume M node m X coordinate
Sub-volume M node m Y coordinate
Sub-volume M node m Z coordinate

Node unit

Sub-volume unit27

３.４.６.４ Variable File Format
Sub-volume 1 node1 variable 1
Sub-volume 1 node2 variable 1
Sub-volume 1 node 3 variable 1：
Sub-volume 1 node n variable 1
Sub-volume 1 node 1 variable 2
Sub-volume 1 node 2 variable 2
Sub-volume 1 node 3 variable 2：
Sub-volume 1 node n variable 2
Sub-volume 1 node 1 variable m
Sub-volume 1 node 2 variable m
Sub-volume 1 node 3 variable m：
Sub-volume 1 node n variable m：：：
Sub-volume M node 1 variable 1
Sub-volume M node 2 variable 1
Sub-volume M node 3 variable 1：
Sub-volume M node n variable 1
Sub-volume M node 1 variable 2
Sub-volume M node 2 variable 2
Sub-volume M node 3 variable 2：
Sub-volume M node n variable 2
Sub-volume M node 1 variable m
Sub-volume M node 2 variable m
Sub-volume M node 3 variable m：
Sub-volume M node n variable m

Variable unit

Sub-volume unit28

３.５. Parameter File
The Parameter file is in an ASCII format. By specifying the file name in a command line
option of the filter program, the parameters are interpretered and set as input parameters.
Table 6 List of filter input parameters
Parameter

Parameter detailDefault

name

Notes

value

in_dirInput file directory“.”

Directory path of input files※１

field_file

fld file name-

Only for structured grid data※2，
3

in_prefix

Input file prefix“input.”Only for unstructured grid data

in_sufix

Input file suffix“.dat”

Only for unstructured grid data

out_dir

Output file directory“.”

Directory path of output files※1

out_prefix

Output file prefix“output.”

FormatStep“%05d”

numberOnly for unstructured grid data

formatstart_step

Start step number

1

end_step

End step number

1n_layerNumber of octree0

An integer from 0 to 7

00: SPLIT format

layer
output_type

File format

1: sub-volume aggregate
2: step aggregate
file_number

Number of output

0

An integer greater than 0.

files

When 0 is set, the number ofsub-volume is used.
Valid

only

in

Sub-volume

aggregate file format.
mpi_volume_div

Numberofparallelism

MPI

1

The

in

total

processes

sub-volume

number
is

of

MPI

given

by

mpi_volume_div × mpi_step_div.
※4

mpi_step_div

Numberof

MPI

parallelism

in

time

1

The

total

processes

stepnumber
is

of

MPI

given

by

mpi_volume_div × mpi_step_div※429

mpi_div

Setting of 2D MPI

2

0: defined by ‘mpi_volume_div’

parallel processingand ‘mpi_step_div’.
1: automatic with priority on
sub-volume decomposition.
2: automatic with priority on step
decomposition.
Options 1 and 2 do not workwhen‘mpi_volume_div’

and‘mpi_step_div’ are set.
multi_elem_type

Flag

on

mixed

element

00: data with a single element type

type

1: data with multiple element

unstructured grid
temp_delete

types

Flag on temporary
files

produced

processing

1

0: keeptemporary filesby

1: delete temporary filemixedelement data
※1. Although directories can be specified either with absolute paths or relative paths, anabbreviationof HOME directory, ~(tilde), cannot be used.
※2. Parameters, ‘nstep’, ’ndim’, ’dim1’, ’dim2’, ’dim3’, ’veclen’, ’coord[123]’, and ’variable’ are referred.
※3. When ‘field_file’ is specified, input data is recognized as structured grid data.
Otherwise, input data is treated as unstructured grid data. When input data is
structured grid data, the output data is converted as unstructured grid data withlinear hexahedral elements in three dimensions and with linear quadrilateral elements
in two dimensions.
※4. When ‘mpi_volume_div’ and ‘mpi_step_div’ are specified, an error occurs if‘mpi_volume_div × mpi_step_div’ does not agree with the number of processes.

３.６. MPI Parallel Processing
This section describes how to define the number of divisions in MPI parallel processing. In
the following, we consider data processing for 50 steps × 8 sub-volumes.
（１） Step preferred paralleliztionWhen the number of processes is equal to or less than the number of steps, divide
all the steps by the number of processes.
Example) When 8 processes are used, each process treats 6 steps × 8
sub-volumes, or 7 steps × 8 sub-volumes.

When the number of processes is larger than the number of steps, the number of30

processes used in the filter program is given by multiples of the number of steps,
and sub-volume parallelization is also used.
Example) When 128 processes are used, the filter program works with 50×2=100 processes (with the residue of 28 processes), and each process treats 1
step × 4 sub-volumes.
（２） Sub-volume preferred parallelization
When the number of processes is equal to or less than the number ofsub-volumes, divide all the sub-volumes by the number of processes.
Example) When 8 processes are used, each process treats 50 steps × 1
sub-volue.

When the number of processes is larger than the number of sub-volumes, thenumber of processes used in the filter program is given by multiples of the numberof sub-volumes, and step parallelization is also used.
Example) When 128 processes are used, the filter program works with 8×
16=128 processes (with the residue of 0 process), and each process treats 6
steps × 1 sub-volume or 7 steps × 1 sub-volume.

（３） User-specified parallelization
When the parallel processing number, ‘mpi_volume_div’ and ‘mpi_step_div’
are specified, an error occurs if ‘mpi_volume_div × mpi_step_div’ does notagree with the number of processes..

３.７. Execution Method in Staging Environment
This section describes how to execute the fliter program in the staging environment on
the K computer. The filter program should be launched with keeping a consistency
between the parameter file and staging parameters. Depending on the output data formatof the filter program, multiple processes may produce a single file. In such a case, the data
output should be made on a shared domain on local file system which is accessible frommultiple processes.

31

３.７.１. Execution Shell Script and Parameter File#!/bin/bash -x##PJM --rsc-list "elapse=01:00:00"#PJM --rsc-list "node=64"#PJM --rsc-list "rscgrp=small"#PJM --stg-transfiles all#PJM --mpi "proc=64"#PJM --mpi "use-rankdir"#Specifing usage of rank directory#PJM --stgin "rank=* ./filter%r:./"#Stage in for load module………①#PJM --stgin "rank=* ./param.txt%r:./"#Stage in for file…………………②#PJM --stgin "rank=0 /data/ucd/ucd*.dat

0:../"#Stage in for shared file…………③#PJM --stgout "rank=* %r:../output*.dat./"#Stage out for resulting file……④#PJM --stgout "rank=* %r:./pbvr_filter.* ./LOG/"#Stage out for file…………………⑤#PJM -S. /work/system/Env_base
export PARALLEL=8
export OMP_NUM_THREADS=8
mpiexec -n 64 lpgparm -p 4MB -s 4MB -d 4MB -h 4MB -t 4MB ./filter param.txt………⑥① Transfer a load module to a rank directory of each process.
② Transfer a parameter file to a rank directory of each process.
③ Transfer input data to a shared domain in local file system.
④ Transfer output data from a shared domain to a directory in global file system.
⑤ Transfer log and error files from a rank directory to a directory in the global file system.
⑥ Launch the load module in a rank directory of each process using the parameter file ina rank directory of each process as an argument.

32#
in_dir=../…………⑦

field_file=pd3d.fld
out_prefix=case0
out_dir=./…………⑧

file_type=0…………⑨n_layer=3
start_step=0
end_step=511⑦ Specify a path of input data. (In the staging environment, the path should be given by arelative path. The above sample reads input data from a shared domain.)⑧ Specify a path of output data. (In the staging environment, the path should be given bya relative path. The above sample writes output data to a rank directory in eachprocess by using of SPLIT File Format.)⑨ Specify an output file format. (The above sample uses the SPLIT format.)

３.７.２. Input-output Files and Directories
This section describes the relation between input-outpu files treated in the filter programand directories in the staging environment. Output data in the SPLIT format can be written
on a rank directory, while output data in the other formats requires a shared directory fordata aggregation.

Table 7 Table of input-output files and directories on K computer
IO
Input

File type

Rank directory

Shared domain

Parameter file

○

※1

○

Input data○※2○

SPLIT format○○

Output

Step

×

○

data

format
×

○

○※3

×

Outputaggregate

Sub-volume
aggregate format
Log & error file
※1. The parameter file is read only from rank 0.

※2. The size and number of the input files should not exceed the resource limit of thestaging environment ( 800 files/node, 14GB/node).
※3. The output directory is fixed to a rank directory.
33

３.８. Unstructured Grid data with Mixed Elements
When unstructured grid data contains several element types, the filter program firstly
generates UCD birary data for each element type, and then divides the UCD binary datawith a sigle element type into sub-volumes, which are read from the PBVR server.
By setting 1 to the papameter ‘multi_element_type’ in the parameter file, the filter
program produces sub-volume files for each element type.
#
in_dir=.
in_prefix=MULTI
in_suffix=.dat
out_dir=.
out_prefix=div
out_prefix=.dat
format=%03
start_step=1
end_step=20
multi_element_type=1

Output files for each element type are generated with file names, which have a 2
digit code representing an element type at the beginning. The following list shows
element names and codes.

34

Table 8 List of element types
Element nameElement type code

Triangle Linear

2

Quadrilateral Linear

3

Tetrahedron Linear

4

Pyramid

5

Prism

6

Hexahedron Linear

7

Triangle Quadratic

9

Quadrilateral Quadratic

10

Tetrahedral Quadratic

11

Hexahedral Quadratic

14

When the input data defined by the above parameter file consists of linear tetrahedral
elements and quadratic tetrahedral elements, the following output files are generated.

Table 9 File names for mixed elements
Original

mixedLineartetrahedral

Quadraticelements data

data

tetrahedral data

MULTI001.dat

04-div001_～

11-div001_～

MULTI002.dat

04-div002_～

11-div002_～

MULTI003.dat⇒

04-div003_～

11-div003_～

MULTI004.dat

Decomposition

04-div004_～

11-div004_～

04-div005_～

11-div005_～

MULTI005.dat：：

MULTI020.dat

04-div020_～

35：
11-div020_～

４ Server
４.１. Overview
The server program reads sub-volume files, which are produced by the filter program,and performs parallel visualization with the PBVR technique to generate particle data as
the result of visualization.

４.２. Launch Server
PBVR can operate on supercomputers both in a batch mode, which generates only
particle data in batch processing, and in a client-server mode, which generates particle
data in interactive processing by connecting the client program and the server program via
socket communications. Stand-alone pricessing on PCs or workstations is also possible bylaunching the client program and the server program in the client-server mode on thesame machine. The following shows how to launch the server.
（launch in MPI+OpenMP processing with N processes）$ mpiexec -n N ./CPUServer（launch only in OpenMP）$./CPUServer※1. As the MPI+OpenMP server operates with master-slave MPI processing, the numberof process N should be specified by the number of slave process + 1.
※2. In both processing modes, the number of OpenMP threads is set in theenvironmental variable‘OMP_NUM_THREADS’.

※3. In Windows, the command is launched from VS2013 x64 Native Tools commandprompt.

Table 10 List of command line options for the server program
Option-hLaunchPossible

Defaultmode

parametersparameters

SB

-

-

Functionshows a list of options and possible
parameters

-B

B

-

-

launch in the batch mode

36

-pa

B

File name-name of the visualization parameter
file

-sl

B

1～99

1

sub-pixel level※2-rlB

1～999991

repeat level※2-plimitB

1～99999

10

limit of the number of particles (106)
※2

-vin

B

File name-name of the pfi file of input volume
data※2-pout

B

File name./name of the output particle data file-p

SPort number

60000

port

numberofsocketcommunication-vieweBr

100 ～ 9999
×100

620×620

viewer resolution～

9999※1. In Launch mode, S, B denote S：client-server mode and B：batch mode.
※2. In the current version, a transfer function in the batch mode is given by thevisualization parameter file, which is specified by the command line option ‘-pa’.
However, the other parametes in the visualization parameter file does not work, andthey should be given by command line options.

４.２.１. Launch in Batch Mode
By specifying the command line option ‘-B’, the server program is launched in thebatch mode. Refer to the following example for launching the server program in the batchmode（MPI+OpenMP.）$ mpiexec –n 5 ./CPUServer –B -vin ./data/case.pfi -pout ./output/case –sl 2 –rl 2 –plimit
10 –pa ./param.in
In the above example, the input data, ’./data/case.pfi’, is processed with thevisualization parameter file, ‘./param.in’, and with command line options, ‘–sl 2 –rl 2 –plimit 10’, to output the following particle data.
./output/case_XXXXX_YYYYYYY_ZZZZZZZ.kvsml
XXXXX：Number of steps（5 digit number）

YYYYYYY：Sub-volume number（7 digit number）

ZZZZZZZ：Total number of Sub-volumes（7 digit number）

37

The visualization parameter file, which is specified by the command line option ‘-pa’, isgenerated by interactive processing in the client-server mode. Large-scale data processingin the batch mode is executed by using this file as it is, or by modifying a part of parametersin it.

４.２.２. Launch in Client-Server Mode
When the command line option ‘–B’ is not specified, the server program is launchedin the client-server mode as shown in the following example.
$ mpiexec -n 5 ./CPUServer
first reading time[ms]:0
Server initialize done
Server bind done
Server listen done
Waiting for connection ...
When “Waiting for connection” appears as the above example and the server
program wait for socket communications from the client program, launch the client
program on another terminal. In the client-server mode, an input volume data name isgiven on the client program.
The default value of a port number used in the socket communication is 60000. To
change the port number, specify the command line option ‘–p’as follows.
$mpiexec -n 5 ./CPUServer -p 55555

４.２.３. Connecting Client and Server by Socket Communication
４.２.３.１ Local Connection
The following example shows how to launch both the client program and the server
program on a sigle machine (machine A). In this example, they cooperate using the default
port number 60000 of machineA.
Step 1[Launch server]
machineA> mpiexec –n 5 ./CPUServer
Step 2[Launch client]machineA> ./PBVRViewer –vin filename38

４.２.３.２ Remote Connection between Two Points
The following example shows how to launch the client program on machine A and theserver program on machine B where the two machines are located at distant places. This
example uses ssh port forwading to connect the 60000 port of machineA to the 60000
port of machine B . The server and client programs on the two machines cooperate
through the default port number 60000. Once the ssh port forwading is established, thelaunching method is basically the same as the stand-alone mode or the local connection. In
Windows, the ssh port forwarding is set up by using ssh softwares such as Tera Term andPutty.
Step1 [ssh port forwarding A→B]
machineA> ssh -L 60000:localhost:60000 username@machineB（Forwarding the 60000 port of machineA to the 60000 port of machineB）
Step2 [Launch server]
machineB> mpiexec –n 5 ./CPUServer
Step3 [Launch client]machineA> ./PBVRViewer –vin filename

４.２.３.３ Remort Connection via Several Machines
This section provides an example of connection between the client program and theserver program on two remote machines A and B via machine C for security or otherreasons. Once the ssh port forwading is established, the launching method is basically thesame as the stand-alone mode and the two point remote concection.
Step1 [ssh port forwarding A→C]
machineA> ssh -L 60000:localhost:60000 username@machineC（Forwarding the 60000 port of machineA to the 60000 port of machineC）
Step2 [ssh port forwarding C→B]
machineC> ssh -L 60000:localhost:60000 username@machineB（Forwarding the 60000 port of machineC to the 60000 port of machineB）
Step3 [Launch server]
machineB> mpiexec –n 5 ./CPUServer
Step4 [Launch client]machineA> ./PBVRViewer –vin filename39４.２.３.４ Test Method for ssh Port Forwading Connection
To check if ssh port forwarding is available, use the following sample program, whichsimply transfer input characters on a server program to a client program. This program isavailable from the link below.
“C for Linux 2” Mitsuyuki Komata, SYUWA System, Inc, September 2005 (Japanese).
http://www.ncad.co.jp/~komata/c4linux2/
How to launch the server program./server port_number
How to launch the client program./client server_hostname port_number

４.２.３.５ Connection with the Pre-post Server of the K-computerThis section shows an example of connecting a PC (machineA) in a laboratory to thedata processing server of the K-computer (Pre-Post server pps3) via the login node of
the K-computer (klogin).
Step1 [ssh port forward A→klogin]machineA> ssh -L 60000:localhost:60000 username@k.aics.riken.jp（Forwarding the 60000 port of machineA to the 60000 port of klogin）
Step2 [ssh port forward K login node→pre-post server]
klogin> ssh -L 60000:localhost:60000 username@pps3（Forwarding the 60000 port of klogin to the 60000 port of pps3）
Step3 [Launch server]
pps3> mpiexec –n 5 ./CPUServer
Step4 [Launch client]machineA> ./PBVRViewer –vin filename（Forwarding the 60000 port of klogin to the 60000 port of pps3）

４.２.３.６ Local Connection on Windows
This section shows how to launch both the client and server programs on a sigle
Windows machine. The VS2013 x64 Cross Tools command prompt in VisualStudio2013
is used as a terminal for launching the program.
Step1 [Launch server]
Windows> CPUServer.exe

40

Step2 [Set a client parameter for Windows]
Windows> set TIMER_EVENT_INTERVAL=1000
Step3 [Launch client]
Windows> PBVRViewer.exe –vin filename
Another launching method is to execute the following batch file.
set TIMER_EVENT_INTERVAL=1000
start PBVR_Server_win.exe
PBVRViewer.exe –vin filename

４.２.３.７ Remote Connection of Windows Client
To connect a Windows machine and a remote server, execute port forwarding by using
SSH client softwares such as TeraTerm or Putty. The following example shows how tomake port forwarding with TeraTerm.
（１） Launch TeraTerm and cancel the “new connection” panel.

（２） Select “Setup → SSH Forwarding” from the menu bar. Select “add” in the“SSH transfer”panel.

41

（３） In the “Select direction for forwarded port” dialogue, select “Forward local port”
and enter the port number to be used on the client program. In the“to remotemachine” box, enter the domain name or the IP address of the server. In the“port”
box, enter the port number to be used on the server. Click on “OK” to complete thesetup of Port forwading.

（４） Connect to the server. Select “File → New connection” from the menu bar. In
the “new connection”panel, enter the host name of the serve and click on “OK”.
In the “SSH Authentication” panel, enter the user name and passphrase, or specify
the location of the private key file, and click on “OK”.

42

The following procedures show how to launch the client and server programs after
establishing port forwarding. This example uses the VS2013 x64 Cross Tools commandprompt in VisualStudio2013 as a terminal for launching the client program.
Step1 [Launch server]
Server> mpiexec –n 4 ./CPUServer –p port_number
Step2 [Set a client parameter for Windows]
Windows> set TIMER_EVENT_INTERVAL=1000
Step3 [Launching the client]
Windows> PBVRViewer.exe –vin filename –p port_number
Note that, the client program on a Windows machine can be launced also by using thefollowing batch file.
set TIMER_EVENT_INTERVAL=1000
PBVRViewer.exe –vin filename –p port_number

43

５ Client５.１. Overview
Client operates in the client-server mode or in a stand-alone mode.
In the client-server mode, the client program receives particle data, which is generated as the resultsof visualization on the server, and renders it using OpenGL. The client program also sets visualization
parameters (Sub-pixel level, Transfer function etc.), which are used in processing volume rendering on
the server program, and sends them to the server. Data transfer between the client program and theserver program is performed using socket communications with an arbitrary port number.
In the stand-alone mode, the client program reads and displays particle data which is generated by
the server program in the batch mode.

５.２. Launching Client
The following examples show how to launch the client program in the client-server mode and in thestand-alone mode. The client program opens five panels, Viewer, Main panel, Transfer function editor,
Time panel, and Animation panel, when it starts successfully.
（Launch in the client-server mode）※1$ ./PBVRViewer -vin [sub-volume file name※2] [command line option]（Launch in the stand-alone mode）$ ./PBVRViewer [particle data file name※3] [command line option※4]※1. The client-server mode should be launched after starting the server program.
※2. The sub-volume file name should be specified by the absolute or relative path of the pfi file.
※3. The particle data should be stored in a directory, which does not contain other kvsml files.
※4. Note that in the stand-alone mode, the particle size is not rendered correctly if it is not specified
by the command line options, the subpixel level（-sl）. and the repeat level（-rl）.
Table 11 a list of command line option of client
Option-hLaunchPossible

Defaultmode

parametersparameters

C, S

-

-

Function
display a list of options and possible
parameter

-sl

C

1～99

1

sub-pixel level

44

-rl

C

1～999991

repeat level

-S

C

u, m

u

particle sampling method（u: uniform sampling,
m:metropolis sampling）-tdataC

all, divallparticle data transfer method（all: step batch transmission,
div: sub-volume divide forwarding）

-vin

C

file name-name of the pfi file of input volumedata-tf※2※３C

file name-name of the transfer function file-pCport number

60000

port

numberofsocketcommunication-viewer

C, S

100 ～ 9999 ×

620×620

viewer resolution-

shading method※3

100～9999-shading

C, S{L/P/B},ka,kd,ks,n※1. In Launch mode, C, S denote C：client-server mode，S：stand-alone mode.

※2. Transfer function files are generated by the Export File button in the “Transfer function
editor” panel.
※3. The transfer function, which is specified in this commandline option, is reflected by pushing“Apply” button in “Transfer function editor”. The transfer function file can be loaded
also by “Import File” button.
※4. This argument specifies shading parameters. The shading process gives lighting effect byemphasizing color tone of the viewing object. The lighting is expressed by the sum of threecomponents: anmbient, diffusion, specular reflection._The parameters specified by theargument modifies intensity of these three components.
L: Lambert Shading
effect：this method calculates the lighting without the specular reflection.
parameters：ka（coefficient for ambient 0〜１），kd（coefficient for diffusion 0〜１）
P：Phong Shading
effect：this method calculates the lighting with the specular reflection which is seen insmooth metal or mirror (also called highlight.)
parameters：ka,kd,ks（coefficient for specular reflection 0〜１），n（strength of highlight ０〜１００）
B：Blinn-Phong Shading
effect：the reduction model of the Phong Shadingparameters：ka,kd,ks,n45

５.３. Terminating PBVR
５.３.１. Standard Termination
The client program renders time-series data from the initial time step to the final time stepand repeats loop operation by returning to the initial time step after the final time step. To
terminate the operation, enter ‘Ctrl+c’in a console in which the client program islaunched.
In the client-server mode, entering‘Ctrl+c’ in the client console terminates both theclient and server programs, when the time step is updated and they are synchronized.
However, the client program ignors the ‘Ctrl+c’ key while client-server communications
are interrupted with “Stop” button in “Time panel”.

５.３.２. Forced Termination
If the server program is terminated with the ‘Ctrl+c’ key, the client program stucks andcannot be terminated with the ‘Ctrl+c’ key. Also, even if the ‘Ctrl+c’ key is pressed toterminate the client program, both the client and server programs might stuck, when thetime step is not updated because of heavy particle generation processes and some otherreasons. In such a case, obtain process IDs of the client and server processes using ‘ps’command, and then, force-quit them with ‘kill’ command as follows.
【Forced termination of the client process】$ ps -C PBVRViewer
PID TTY
19582 pts/6

TIME CMD
00:00:00 PBVRViewer$ kill -9 19582【Forced termination of the server process】$ ps -C CPUServer
PID TTY
19539 pts/5

TIME CMD
00:00:00 CPUServer$ kill -9 19539

46

５.４. Operating Instructions
５.４.１. Viewer
Figure 6 shows “Viewer” displaying a rendering result of particle data.

Figure 6 Viewer【Operation】
Rotation：move a mouse while left-clicking
Translation：move a mouse while right-clicking
Zoom：scroll a mouse wheel up and down, or move a mouse up and down while holding
down the ‘Ctrl’ key【Display】
time step：time step of displayed data
fps：frame rates（frame/sec）

47

５.４.２. Main Panel
Figure 7 shows a main panel of the client program. Each item in the panel is described
below.

Figure 7 Main panel
・SUB PIXEL LEVEL
Specify the subpixel level value Ls which controls the image resolution.
・REPEAT LEVEL
Specify the repeat level value Lr which controls the image resolution.
Under the condition of Lr=Ls2, almost the same image resolution is obtained, andtherefore, the parameters (Ls,Lr)=(2,1) and (Ls,Lr)=(1,4) give almost the same
picture. When (Ls,Lr)=(2,1), the rendering process is executed at a time. However,when (Ls,Lr)=(1,4), the rendering process is devided into four steps and ensemble
average of image buffers is performed before displaying the image. As image48

buffer sizes are proportional to Ls2, it is recommend to use Lr when the memory
size is limited in the rendering process.
・PARTICLE LIMIT
Specify the maximum number of particles, which are generated in the server
program, in the unit of M(106) particles to avoid explosive increase of the particle
number due e.g. to false setting of a transfer function or any other reason.
・EYE POINT
Specify a camera’s eye point.
・CENTER POINT
Specify a camera’s center poit.
・UP VECTOR
Specify a camera’s up vector.
・RESOLUTION
Specify viewer’s resolution.
・SENDING
Show the progress of data send to the server program.
・RECIEVINGShow the progress of data recievr from the server program.
・CPU MEMORY
Display the system memory usage (unit: MB).
・GPU MEMORY
Display the GPU memory usage (unit: MB). Active when GPU rendering is used.
・PBVR Filter information
Display information about the volume data on the server program (contents in thepfi file).
・CROP button（the current version support this function only on Linux and Windows）
Display “CROP panel”. Details of “CROP panel” is described in the next section.
・Set parameter button
Send parameters specified in “Main panel” to the server program.

49

５.４.３. CROP Panel“CROP panel”appears by pressing “CROP button” in “Main panel”, and is related tooperations of extracting and rendering elements involved within a region of interest (ROI),
which is specified with a given shape. A shape of a ROI can be selected from cuboid,
sphere, and cylinder. Note that “CROP panel” is not available on Mac in the current
version.

Figure 8 CROP panel
Select Object：Specify a shape of a region of interest (ROI)
CROP：Cuboid region
SPHERE：Sphere
PILLAR-XY base：Cylinder with a XY base
PILLAR-YZ base：Cylinder with a YZ base
PILLAR-XZ base：Cylinder with a XZ base

50

CROP：Specify a range of the cuboid
SPHERE：Specify the center and radius of the sphere
PILLER：Specify the radius, height, and center coordinate value of the cylinder
RESET：Reset CROP panel
APPLY：Extract a ROI
CLOSE：Close the panel
By displaying this panel, the shape of ROI is overwritten in “Viewer”as follows.

Feature 9 Viewer panel when Crop feature is applied51

５.４.４. Transfer Function Editor“Transfer Function Editor” creates a transfer function defining the correspondence
between scalar data and color/opacity in volume rendering. In standard volume rendering,a transfer function is defined by a single physical quantity. However, in PBVR, a new
multi-dimensional transfer function is implemented with the following three features:（１） Assign independent variable quantities to color and opacity.
（２） Define each variable quantity by an arbitrary function of the coordinates X, Y, Z,
and variables q1, q2, q3….
（３） Multiple one-dimensional transfer functions t1-t5 are synthesized with anarbitrary function to define a multidimensional transfer function.
The newly developed transfer function design realized extremely flexible visualization.
Figure 10 shows “Transfer Function Editor”. Each item in the panel is described below.

Figure 10 Transfer Function Editor
・Transfer Function RESOLUTION Box

52

Specify resolution of a transfer function
・Transfer Function SYNTHESIZER Box
Specify a function to synthesize one dimensional transfer functions t1～t5 ※1・Transfer Function Name Pull-down Menu
Select a transfer function (t1-t5) to edit.
・Reset Button
Reset the panel.
・Apply Button
Send a transfer function created in this panel to the server.
・File Path Box
Specify a file path for saving and loading a transfer function file.
・Export File Button
Save a transfer function created in this panel to a file in the same format as aparameter file specified with the command line option‘-pa’.
・Import File Button
Load a transfer function stored in a file to this panel.

５.４.４.１ Color Map Editor【Transfer Fuction Color Map Category】
Define the correspondence between a variable quantity and color for a transfer function
selected in “Transfer Fuction Name”.
・Color Bar
Display a color bar of a transfer function created in this editor.
・Variable Box
Specify the definitional equation of a (synthesized) variable quantity used for colorof the selected transfer function. The following variables can be used in theequation.
Physical quantities：q1, q2, q3,・・・qn
Coordinates：X, Y, Z
・Range Min Box
Specify the minimum value of the variable quantity.
・Range Max Box
Specify the maximum value of the variable quantity.
・Server side range min Box
Display the minimum value of the (synthesized) variable quantity obtained in theserver program.
・Server side range max Box
Display the maximum value of the (synthesized) variable quantity obtained in theserver program.

53

・Color Map Editor (freeform curve) Button
Display a sub-panel which can create a transfer function with freeform curves
drawn by a mouse.

Figure 11 Color Map Editor (freeform curve) panel
・Color paletteSpecify saturation ,brightness, and hue of a color by placing a mouse. In the left
space, the horizontal axis shows saturation while vertical axis shows brightness.
The neighboring bar shows color hue.
・RGB barSpecify a hue of a color by placing a mouse. The upper-right box displays a color
created by the Color palette and the RGB bar.
・Color
Overpaint the bar with a color created by the Color palette and the RGB bar bytracing the color bar with a mouse while left-clicking. The blending ratio of the
background color and overpainting color is determined by the mouse’s vertical
position. For example, when the upper side of the color bar is traced side to side,
the traced area is painted only by the specified color; while the middlel part
between upper and lower sides of the color bar is traced side to side, 50% of thespecified color and 50% of the existing color are mixed in the traced area.
・Reset Button
Reset the panel.
54

・Undo Button
Undo the last action made by a mouse.
・Redo Button
Redo the last action undone by a mouse.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.
・Color Map Editor (expression) Button
Display a panel which can create a transfer function in a functional expression.

Figure 12 Color Map Editor (expression) panel
・Color
Display a color bar of a transfer function created in this panel.
・R BoxDescribe a transfer function of the R component of the color.
・G BoxDescribe a transfer function of the G component of the color.
・B BoxDescribe a transfer function of the B component of the color.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.
・Color Map Editor (control points) Button
Display a panel which can create a transfer function by specifying control points.

55

Figure 13 Color Map Editor (control points) panel
・Color
Display a color bar of a transfer function which is created in this panel.
・Control Point
Specify values of control points (up to 10 points) in the CP1)-10) boxes.
・Red
Describe the R component of the color at the control points.
・GreenDescribe the G component of the color at the control points.
・Blue
Describe the B component of the color at the control points.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.
・Color Map Editor (select colormap) Button
Display a panel which can create a transfer function by selecting color bar
templates.

56

Figure 14 Color Map Editor (select colormap) panel
・Color
Display a color bar of a transfer function which is created in this panel.
・Default Color pull-down Menu
Select a color bar to be set as a transfer function from the following options.
RainBow
Blue-white-red
Black-red-yellow-white
Black-blue-violet--yellow-white
Black- yellow-whiteBlue-green-red
Green-red-violet
Green- blue--white
HSV model
Gray-scale
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.

５.４.４.２ Opacity Editor【Transfer Fuction Opacity Map Category】
Define the correspondence between a variable quantity and opacity for a transfer
function selected in “Transfer Fuction Name”.
・Opacity Bar
Display a transfer function curve created in this editor.
・Variable Bar

57

Specify the definitional equation of a (synthesized) variable quantity used for
opacity of the selected transfer function. The following variables can be used in
the equation.
Physical quantities：q1, q2, q3,・・・qn
Coordinates：X, Y, Z
・Range Min Box
Specify the minimum value of the variable quantity.
・Range Max Box
Specify the maximum value of the variable quantity.
・Server side range min Box
Display the minimum value of the (synthesized) variable quantity obtained in theserver program.
・Server side range max Box
Display the maximum value of the (synthesized) variable quantity obtained in theserver program.
・Color Map Editor (freeform curve) Button
Display a panel which can create a transfer function with freedom curves drawn
by a mouse.

Figure 15 Opacity Map Editor (freeform curve) panel
・Opacity
Create a transfer function of opacity. A freeform curve is drawn by clicking the left
mouse botton and dragging the mouse. A linear interpolation line between two
points is drawn by specifying control points with right clicks.
・Reset Button
Reset the panel.
・Undo Button
Undo the last action made by a mouse.
・Redo Button
58

Redo the last action undone by a mouse.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.
・Color Map Editor (expression) Button
Display a panel which can create a transfer function in a functional expression.

Figure 16 Opacity Map Editor (expression) panel
・Opacity
Display a transfer function of opacity following a functional expression given in the
O box.
・O BoxDescribe a functional expression of a curve which specifies a transfer function of
opacity.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.
・Color Map Editor (control point) Button
Display a panel which can create a transfer function by specifying control points.

59

Figure 17 Opacity Map Editor (control points) panel
・Opacity（upper bar）
Display a transfer function of opacity created in this panel.
・Control Point
Specify values of control points (up to 10 points) in the CP1)-10) boxes.
・Opacity（right colomn）
Specify opacity at the control points.
・Save Button
Save a transfer function created in this panel.
・Cancel Button
Close the panel.

60

５.４.４.３ Function Editor
Built-in functions available in the function editor which is used to synthesize transfer
functions and variable quantities, and to define color map curves, and opacity curves arelisted below.

Table 12 Operation available in function editor
Operation

Form

+

+

-

-

×

*

/

/

Sin

sin(x)Cos

cos(x)Tan

tan(x)Log

log(x)Exp

exp(x)

Square rootsqrt(x)

Power

x^y

61

５.４.５. Time panel
Visualization time step is controlled by “Time panel” in Figure 18. Each widget works asfollows.

Figure 18 Time panel
・Progress
Show the progress of time step by percentage.
・Time step
Specify the time step for rendering.
・Min Time
Specify the minimum time step for ROI.
・Max Time
Specify the maximum time step for ROI.
・Start/Stop
Start or Stop the communication between the client and server programs.

62

５.４.６. Animation panel
Image files displayed in “Viewer” are saved as sequential number image files for each time
step. The file format is BMP. The sequential number image files can be converted andcompressed into movie files like mpeg using freewares such as ImageMagic and ffmpeg.

Figure 19 Animation panel

mode：Only image files are available.
capture： Sequential number image files for each time step are produced when turned on.
image file：Specify the prefix of the sequential number image files.
The sequential number image files are stored in the client directory with the following
names.
PBVR_image.00001.bmp
PBVR_image.00002.bmp

63

６ Examples Using Sample Data
The following sections show the functions of PBVR system by using the sample data
‘gt5d.tgz’.

６.１. Filter Processing
By extracting ‘gt5d.tgz’, the following files are generated under ‘./gt5d’.
gt5d.fld

AVS field fileco3d.dat

coordinate data

pd3d.datvariable１

psid.datvariable２

param.txt

filter program input parameterdemo1.tf

transfer function file for demonstration1

demo2.tf

transfer function file for demonstration2

Execute the filter program with the following command（OpenMP ver.）$./filter ./gt5d/param.txt
The contents of param.txt are shown in the following.
#
in_dir=./field_file=gt5d.fld
out_dir=./
out_prefix=casestart_step=0
end_step=4
The above example specifies the SPLIT file format by default, a single sub-volume (nosub-volume decomposition), and the same directory both for input and output. This filter
process generates the following files in the specified directory.
case.pfi

pfi file

case_YYYYYYY_ZZZZZZZ_connect.dat

element configuration filecase_YYYYYYY_ZZZZZZZ_coord.dat

node coordinate filecase_XXXXX_YYYYYYY_ZZZZZZZ.kvsml

kvsml file64

case_XXXXX_YYYYYYY_ZZZZZZZ_value.dat

variable file

６.２. Starting PBVR
[ step１]

Launch server（OpenMP ver.）$./CPUServer
first reading time[ms]:0
Server initialize done
Server bind done
Server listen done
Waiting for connection ...
[step２]

Launch client. This example uses the metropolis sampling, the sub-pixel level

and the repeat level are set as (Ls,Lr)=(4,2), and the particle limit is given as 100M particles.
$./PBVRViewer -S m -vin ./gt5d/case.pfi -plimit 100 -rl 2 -sl 465

Figure 20 Initial state of GUI of PBVR system

66

６.３. Designing Transfer Function
This section shows visualization examples of ‘gt5d.fld’using the multi-dimensionaltransfer function, that is produced by the advanced transfer function design capability of
PBVR. ‘gt5d.fld’ is structured grid volume data consisting of two variables.
※1. Transfer functions used in section 6.3.1〜6.3.3 are stored in ‘demo1.tf’.
※2. A synthesized transfer function used in section 6.3.4 is stored in ‘demo2.tf’.

６.３.１. Volume Rendering of Single Variable
First of all, the volume rendering result for the variable q1 is confirmed by setting thetransfer function t1 as shown in Figure 21. The transfer function is designed using“Transfer Function Editor”. The left side of this panel is color editor and the right side is
opacity editor. This example shows the normal volume rendering using the single variable
q1 for both color and opacity.

Figure 21

Volume rendering result for the variable q1.

67

６.３.２. Multivariate Volume Rendering
The next example shows the result of multivariate volume rendering, in which thevariables q1 and q2 are synthesized as shown in Figure 22. In this example, the color isdesigned by the variable q1, while the opacity is given by the variable q2. The opacity map
extracts two torus surfaces, which are given by iso-surfaces of the variable q2, and thecolor means distributions of the variable q1 on these iso-surfaces.

Figure 22 Multivariate volume rendering result, which maps the color of the variable q1
on the iso-surfaces of the variable q2.

６.３.３. Slice Plane
The slice plane can be extracted by applying the multivariate volume rendering as shownin Figure 23. In PBVR, an arbitorary function can be used to design the transfer function. In
this example, the cylindrical surface X^2+Z^2=const. is extracted and the color of thevariable q1 is mapped on it.

68Figure 23 The slice plane by using the multivariate volume rendering69６.３.４. Synthesis of Transfer Functions
This section explains the transfer function synthesizer of PBVR. Figure 24 shows thetransfer function t4 whose opacity function makes the region for Y>0 transparaent. By
synthesizing the previously described transfer functions t1, t2, t3 and the new transfer
function t4 as (t1+t2)*t4+t3, sub-regions are extracted. In this example, the colors of t2
and t3 are set as （R,G,B）＝（0,0,0）, while the color of t4 is set to be（R,G,B）＝（1,1,1）.
By using the above synthesis equation, the final color becomes the rainbow defined for t1.
On the other hand, the opacity of t4 is multiplied to the sum of t1 and t2 to extract thelower half (Y<0) region of t1 and t2, and then, the the resulting region is synthesized withthe cylindrical surface given by t3. Although PBVR is equipped with “CROP panel”, thetransfer function synthesizer is more flexible because it can extract different ROIs for eachvariable quantity, simultaneously. .

Figure 24 Transfer function synthesizer

70

６.４. Preserving Result
After the design of the transfer function, PBVR system can preserve the resulting image
and parameters in following three ways.
（１） Output image
In order to output the resulting images, select ‘on’ from the drop down menu
‘capture’in“Animation

panel”.

The

bitmap

image

files

(PBVR_image.xxxxx.bmp) are generated in working directory.
（２） Transfer function file
In order to preserve the transfer function file, write file name of the transfer
function in ‘File Path’ box in “Transfer Function Editor” and press ‘Export
File’. This file is loaded by ‘Import File’. Note that the input transfer function is
not reflected until ‘Apply’ is pressed.
（３） Visualization parameter file
For the server program in the batch mode, all the visualization parameters
including the transfer function can be exported. Open “File Sub-panel” from
“Main panel”, specify the parameter filename in it, and press ‘Export File’.

Figure 25 Export visualization results

71

６.５. Example of Batch Mode
This section explains the batch mode server processing using the visualization parameter
file exported in the previous section. This mode is developed for massively parallel
processing on supercomputer. It is useful also for high speed rendering of time series datain the stand-alone mode client processing, in which latency due to particle generation andparticle data transfer can be eliminated.

[Step１]

Launch server in the batch mode（OpenMP ver.）※1$./CPUServer –B –vin ./gt5d/case.pfi –pout ./particle/ -S m -plimit 100 -sl 4 –pa ./param.in[Step２] Launch client in the stand-alone mode.※2$./PBVRViewer ./particle/ -sl 4※1. In the current version, the server program in the batch mode requires commandline options for visualization parameters except for the transfer function.
※2. The sub-pixel level and the repeat level should be specified also in launching theclient in the stand-alone mode because they determine the size of particles in
the redering process.

72