In-Situ PBVR (v.2.2)
User Guide

Nov. 2023
Japan Atomic Energy Agency
Center for Computational Science & e-Systems

Revision Record
VersionDate revised

numberRevised

Revised content

chapter

1.0

2019.03.28-

Release1.1

2021.05.01

Modified to common GUI with Client-Server PBVR.

1.2

2022.03.30

Added polygon composition function

2.0

2023.01.01

Source codes of CS/IS-PBVR are integrated.
Pre-defined color maps are updated.

2.1

2023.04.20

Supported element type is added for unstructured grid
liblary.

i

Contents
１ Introduction...................................................................................................................................... 4

１.１. Overview of Particle Sampler .................................................................................................... 5
１.２. Overview of Daemon................................................................................................................. 5
１.３. Overview of PBVR Client .......................................................................................................... 6
１.４. Operating Environment ............................................................................................................. 6
２ Package Configuration ................................................................................................................... 8

２.１. Load Module Package .............................................................................................................. 8
２.２. Source Code Package .............................................................................................................. 9
３ Build from Source Codes ......................................................................................................... 11

３.１. Daemon and Particle Sampler ................................................................................................ 11
３.２. PBVR Client ............................................................................................................................ 12
４ Setting up of In−SituVisualization ................................................................................................ 13

４.１. Setting Environment Variables ................................................................................................ 13
４.２. Setting of Visualization Parameters ........................................................................................ 13
４.３. Port Forwarding Connection ................................................................................................... 13
４.３.１. Remote Connection between Tow Machines .................................................................. 13
４.３.２. Remote Connection with Sseveral Machines ................................................................. 14
４.４. Launching Daemon and Port Forwarding ............................................................................... 14
５ Particle Sampler ............................................................................................................................ 15

５.１. Particle Generation Function for Structured Grid .................................................................... 15
５.２. Particle Generation Function for Unstructured Grid ................................................................ 16
５.３. Particle Generation Function for AMR Grid ............................................................................. 17５.４. Connection into Simulation Code ............................................................................................ 18
５.４.１. Domain Information and Particle Generation Function ................................................... 19
６ PBVR Client.................................................................................................................................... 20

６.１. Launching PBVR Client........................................................................................................... 20
６.２. Terminating of PBVR Client ..................................................................................................... 20
６.２.１. Forced Termination ......................................................................................................... 21
６.３. Using PBVR Client GUI........................................................................................................... 22
６.３.１. Viewer ............................................................................................................................. 22
６.３.２. Tool Bar ........................................................................................................................... 24
６.３.３. Transfer Function Editor.................................................................................................. 30
６.３.４. Time panel ...................................................................................................................... 46
６.３.５. Particle and Polygon composition ................................................................................... 47ii

６.３.６. Image file production....................................................................................................... 51
６.３.７. Legend panel .................................................................................................................. 56
６.３.８. Viewer Control Panel ...................................................................................................... 58
７ How to Execute Examples ............................................................................................................ 59

７.１. Login node .............................................................................................................................. 59
７.２. Interactive Job ......................................................................................................................... 60
７.３. Windows.................................................................................................................................. 62iii

１ Introduction
This document is a user guide for In-Situ PBVR, which is an in-situ remote visualization systemdeveloped at the Center for Computational Science & e-Systems in Japan Atomic Energy Agency. In
today's supercomputers, the computation speed greatly exceeds the I/O speed, making it difficult to
output computation results. As a result, it has become difficult to apply the conventional visualization
method, in which calculation results on a remote storage are transferred to a PC at hand for visualization.
In-Situ visualization avoids large-scale data I/O by having the visualization program coupled to thesimulation and processed simultaneously in the same environment as the simulation to reliably generatevisualization images. In-Situ PBVR is a framework that enables interactive visualization through particlebased visualization and interactive in-situ control technology, whereas interactive in-situ visualization isdifficult to achieve with conventional polygon-based methods. In-Situ PBVR is a C++ library built using
the Particle Based Volume Rendering (PBVR) method, a particle-based volume rendering method
developed by Koyamada Laboratory, Kyoto University, and the visualization library KVS. The framework
consists of three components: a particle sampler, a daemon, and a PBVR client (Figure １-1).

（１）Particle Sampler

The particle sampler is a visualization library that is coupled to a simulation code to generateparticles in the same environment as the calculation. To build a simulation + in-situ visualization
code, an array of calculation results is passed to the visualization functions provided by the particle
sampler and inserted into the simulation code. The particle sampler refers to the visualization
parameter file on the storage, converts the calculation results of each time step into compressed
particles for visualization, and outputs them on the storage. The particle files are output from eachprocess in a distributed manner.
（２）

DaemonThe daemon runs on the login node or the interactive job and mediates between the particle files

on the storage and the visualization parameters sent from the PBVR client. The daemon monitors
the files on the storage, aggregates the distributed output particle files, and transfers them to theuser PC via the network. The daemon also receives the visualization parameters sent from the
PBVR client and outputs them as a visualization parameter file to be referenced by the particle
sampler. Since this operation is executed asynchronously with the simulation, interactive control ispossible without interfering with the simulation.
（３）

PBVR Client
The PBVR Client runs on the user's PC and provides a viewer to display the visualization results

and a GUI to edit the visualization parameters. PBVR Client communicates with the daemon using
port forwarding. The PBVR client receives the particle data from the daemon, displays the volume
rendered image on the viewer, and sends the visualization parameters edited by the user on the
GUI to the daemon.

4

In-Situ PBVR implements a multivariate visualization function that can be used universally for various
simulations. In 3D data visualization, colors and opacity are assigned to the physical values of thesimulation results, and the function that describes the relationship between them is called the transfer
function. In conventional visualization, 1D transfer functions that assign color and opacity to a single
physical value are used. However, it is difficult to design a multidimensional transfer function for
multivariate data using conventional visualization methods. The In-Situ PBVR provides a transfer
function editor that allows users to design multidimensional transfer functions using algebraic
expressions. The transfer function editor allows users to edit the 1D transfer function of each physicalvariable, and to write a multidimensional transfer function using arbitrary algebraic expressions withthese 1D transfer functions as variables. In this algebraic expression, basic operators, elementaryfunctions, and differential operators can be used.
Supercomputer
Simulation

Network

DaemonParticle Sampler

Aggregation

Particle data

Vis.
Param.

Vis.
Param.
Particle
data

User
PC

User

PBVR Client
ViewerVisualization Param.
Color

Particle data

Opacity

Storage

Figure １-1 The configuration of In-Situ PBVR framework

１.１. Overview of Particle Sampler
The particle sampler is parallelized by hybrid of MPI/OMP programming model, and further
accelerates the generation of particle data by using SIMD operation. The particle sampler is MPI
parallelized without changing the domain decomposition of simulation, and generates particles inelement parallel by OpenMP in each decomposed domain. The synthesis of physical values, thesynthesis of transfer func tions, and the interpolation of physical values, which are required in
multivariate visualization, are vectorized using SIMD operations.
Three types of particle samplers are available according to the simulation grid types, one for structured
grids, one for unstructured grids, and one for hierarchical grids. The particle sampler supports simulation
code written in C/C++ and FORTRAN. For in-situ visualization, the particle sampler is inserted into thetime step loop of the simulation. At this time, multivariate data of simulation results, lattice data, andglobal coordinates of the region are input as arguments of the particle sampler. Also, the memory layoutof multivariate data assumes an array structure (SOA).

１.２. Overview of Daemon
The daemon is executed on an interactive node or interactive job of the supercomputer, and is a key
to realize interactive visualization during batch processing. The daemon constantly monitors files on5

storage, and collects particle files output by the particle sampler. Because this collection is asynchronouswith the particle sampler and the simulation, it does not impede the performance of the simulation. The
collection of the particle files is parallelized by OpenMP, and the daemon aggregates the particle files
into single particle data and sends it to the PBVR client. The daemon also receives the visualization
parameters sent from the PBVR client and updates the visualization parameter file on the storage usedby the particle sampler.
In the in-situ PBVR, since the daemon and the PBVR client send and receive data by socket
communication via the Internet, the interactive node and the user PC need to be connected by port
forwarding. Therefore, if the port forwarding is not permitted on a supercomputer, the daemon does notwork and interactive visualization can not be used. In addition, interactive visualization can not be usedeven on a super computer that employs staging I/O where file output does not occur until the end ofbatch processing.

１.３. Overview of PBVR Client
The PBVR client is lunched on the user PC, and is comprised of a screen for displaying thevisualization result, and a transfer function editor. In the transfer function editor, a “transfer function
synthesizer” (TFS) is implemented in which is a function for realizing multivariate visualization. The TFS
has a volume data synthesisis function which combines variables included in the result data to generate
new volume data, and a transfer function synthesis function which combines multiple transfer functions.
The user specifies the synthesis function by an algebraic expression on TFS. The algebraic expression
is transferred to the PBVR sampler as a visualization parameter, and the volume data and transfer
function synthesis are performed in real time by computer algebra system. Users can use initialmathematical functions and spatial derivatives of physical variables as algebraic expressions on TFS,
and flexibly design the transfer function for the multivariate data by mathematical expressions.

１.４. Operating Environment
In-Situ PBVR is a cross-platform program and can be run on various supercomputers such as Fugaku,
Oakforest PACS at the University of Tokyo/Tsukuba University, and SGI8600 owned by JAEA. The tablebelow shows the operating environment.
Table １-1 PBVR Client
PlatformOS

Compiler

Linux

Ubuntu18

g++

Mac

OSX12

clang

Windows

Windows10

MSVC

Table １-2 Daemon
PlatformOS

Compiler

Linux

RedHatg++

6

Table １-3 Particle Sampler
PlatformOS

Compiler

FUGAKUA64FX (ARM)

Fujitsu compiler

OakforestPACS

Intel Xeon Phi (Knights Landing, KNL)

Intel compiler

SGI8600

Intel Xeon GoldIntel compiler

7

２ Package Configuration
The source code and load module (binary) packages of the In-Situ PBVR are available from the CCSE
public page ( https://ccse.jaea.go.jp/software/); the PBVR client, daemon, and particle sampler are built
independently and work together on the user PC, login node, and computation node, respectively.

２.１. Load Module Package
For the load module, we provide PBVR clients built for Windows, Linux, and Mac, and the daemon
and the particle sampler built for a Linux server. The particle sampler and daemon are built on the
supercomputer SGI8600 and Fugaku (clang-mode). The particle sampler consists of the following threelibraries.
1.

Particle generation library that provides particle generation functions

2.

KVS library that provides visualization functions

3.

Mathematical expression processing library that provides algebraic expression processing
functions

In addition, the particle generation library supports three types of simulation lattice structures(structured and unstructured lattice, and hierarchical lattice). The table below shows the list of load
modules.
Table ２-1 Load modules for PBVR client
Machine type

Parallelization

Package name

Linux

OpenMP

pbvr_client_linux.tar.gz

Mac

OpenMP

pbvr_client_mac.tar.gz

Windows

OpenMP

pbvr_client_win.zip

Table ２-2 Daemon
Machine type

Parallelization

Package name

SGI8600

OpenMP

pbvr_daemon_s86.tar.gz

FUGAKU

OpenMP

pbvr_daemon_fugaku.tar.gz

Table ２-3 Load modules for particle sampler
Machine type

Parallelization

Package name

SGI8600

MPI+OpenMP

pbvr_sampler_s86.tar.gz

FUGAKUMPI+OpenMP

pbvr_sampler_fugaku.tar.gz

8２.２. Source Code Package
By compiling the source code package in user’s environment, particle sampler, daemon, and PBVR
client are generated. A source tree of In-Situ PBVR package are shown in Table ２-4. The source code
package contains particle sampler coupled test simulation code.
Table ２-4 The source tree of In-Situ PBVR
Directory/File name

Explanation

is_pbvr/

Root directory for In-Situ PBVR|-pbvr.conf

Configuration file for the Makefile|-Makefile

Makefile for particle sampler, daemon, and PBVR
client|-arch/Setting files for various environments|-Client/

PBVR client program|-Common/Protocol, communication and common library|-Daemon/

Daemon program|-Example/

Samples of test simulation code|C/

C version|-Hydrogen_struct/for structured grid|-Hydrogen_AMR/

for AMR grid|-Hydrogen_unstruct_tetra/

for unstructured tetrahedral grid|-Hydrogen_unstruct_hex/

for unstructured hexahedral grid|-Hydrogen_unstruct_prism/

for unstructured prism grid|-Hydrogen_unstruct_pyramid/

for unstructured pyramid grid|-Hydrogen_unstruct_quadtetra/

for unstructured quadratic tetrahedral grid|-Hydrogen_unstruct_quadhex/

for unstructured quadratic hexahedral grid|-Fortran/

Fortran version|-Hydrogen_struct/for structured grid|-Hydrogen_AMR/

for AMR grid|-Hydrogen_unstruct

for unstructured grid|-FunctionParser/

Computer algebra library|-InSituLib/

Particle generation library|-struct/for structured grid|-AMR/

for AMR grid|-unstruct/for unstructured grid|-KVS/

KVS library910３ Build from Source Codes
The PBVR client is generated on the user PC, while the daemon and particle sampler are generated
by building the source code on the supercomputer.

３.１. Daemon and Particle Sampler
This section shows the procedure to build the daemon and particle sampler libraries from the source
package. In the following procedure, we assume that the source package has already been downloadedon the supercomputer and the file download path is $HOME.
①

Unzip pbvr_inSitu_1.10.tar.gz$ cd $HOME$ tar xvfz pbvr_inSitu_1.10.tar.gz②

Edit a config file.

The compiler used for the build is controlled by editing pbvr.conf to suit the environment. Table 3 1
gives an overview of a variable specified in pbvr.conf, and Table 3 2 lists the compilation configuration
files that can be used as variable values.
Table ３-1 Variable in pbvr.conf
Variable

Input

Explanation

PBVR_MACHINE

StringSpecify a compile configuration file under arch

Table ３-2 Setting files for compilation
Filename

Explanation

Makefile_machine_gcc

Setting of sequential version compilation with gcc

Makefile_machine_gcc_omp

Setting of OpenMP version compilation with gcc

Makefile_machine_gcc_mpi_omp

Setting of MPI+OpenMP version compilation with gcc

Makefile_machine_intel

Setting of sequential version compilation with intel

Makefile_machine_intel_omp

Setting of OpenMP version compilation with intel

Makefile_machine_intel_mpi_omp

Setting of MPI+OpenMP version compilation with intel

Makefile_machine_fugaku_clang

Setting of compilation on FUGAKU (clang-mode)

Makefile_machine_fugaku_trad

Setting of compilation on FUGAKU (trad-mode)

Makefile_machine_s86_mpi_omp

Setting of compilation on SGI8600
Intel compiler and mpt library.

11

$ cd $HOME/is_pbvr$ cat pbvr.conf
PBVR_MACHINE=Makefile_machine_gcc_mpi_omp
PBVR_MAKE_CLIENT=0
The daemon and the particle sampler libralies are built by make.

③$ cd $HOME/is_pbvr$ make all_clean$ makeTable ３-3 Load modules of daemon and particle sampler
Directory

Load module

Explanation

KVS

libkvsCore.a

KVS

library

providingparticleformatandvisualization functions
CommonlibpbvrCommon.a

Communication library providing protocol for socket
communication

Daemonpbvr_daemon

DaemonFuctionParser

libpbvrFunc.a

Computer algebra library

InsituLib/struct

libInSituPBVR.a

Particle sampler

InsituLib/unstruct

libInSituPBVR.a

Particle sampler

InsituLib/AMR

libInSituPBVR.a

Particle sampler④

This completes the building of the daemon and particle sampler, and we will continue with thebuilding of the test code under Example.

$ cd $HOME/is_pbvr/Example/C/Hydrogen_struct$ make３.２. PBVR Client
The

PBVR

client

is

built

with

the

same

code

as

the

client-server

PBVR(https://ccse.jaea.go.jp/software/PBVR/). To build the PBVR client for In-Situ, edit qtpbvr.conf as follows
to enable PBVR_MODE=IS.
#PBVR_MODE - Either CS (ClientServer), or IS (Insitu) - Needed on all
platforms#PBVR_MODE = CS
PBVR_MODE = IS
For details on how to build a PBVR client, please refer to the manual for the client-server PBVR.

12

４ Setting up of In−SituVisualization
The collaboration of the particle sampler coupled to the simulation, the daemon operating on theinteractive node, and the PBVR client on the user PC interactively visualizes the batch processed
simulation. To accomplish this, some simple configuration and port forwarding connections are required.

４.１. Setting Environment Variables
The daemon and particle sampler use the environment variables shown in Table 41 below, and at
runtime you need to set the environment variables with the export command.
Table ４-1 Environment variables referenced by daemon and particle sampler
Env. var.

Explanation

VIS_PARAM_DIR

Directory placed the transfer function file (visualization param.)※1

PARTICLE_DIR

Destination directory for the particle files generated by particle sampler※1

TF_NAME

File name of the transfer function (without extension)※2※１ If not specified this, the daemon and particle sampler will search the current directory in which
each is running.
※２ If not specified this, daemon and particle sampler adopt default.tf as the transfer function name.

４.２. Setting of Visualization Parameters
When the particle sampler converts volume data into visualization particle data, visualization
parameters such as a transfer function, a range of physical values to be visualized, and a screen
resolution are required. Each item of the visualization parameters are described on a tag basis in thetransfer function file. The user can use default.tf stored in the example of the source package as thetransfer function file at first startup.
In order to execute In-Situ PBVR, the user needs to place the transfer function file in the directory
specified by VIS_PARAM_DIR in Table ４-1 with the file name specified by TF_NAME.
The user can edit the contents of the transfer function file by the GUI by activating the daemon and
the PBVR client. The transfer function file specified by the environment variable is read by the daemon,
displayed on the PBVR client, and overwritten after editing.

４.３. Port Forwarding Connection
The particle data and visualization parameters are sent and received by socket communication
between the daemon and the PBVR client. In order to perform socket communication between theinteractive node on the remote supercomputer and the local PC, the user needs to connect both ports
by ssh port forwarding.

４.３.１. Remote Connection between Tow Machines
The following example shows how to connect local machineA to remote machine using port
forwarding.
13

machineA> ssh -L portnumA:hostnameB:portnumB username@machineB
In the above command, portnumA is the port number of machineA, hostnameB is the host name ofmachineB, and portnumB is the port number of machineB. hostnameB is often displayed on the terminalof machine B, and can also be confirmed by the hostname command. Also, if port forwarding is permitted
on the login node of machine B, there is no need to enter a special host name for hostname B, but
localhost can be used.

４.３.２. Remote Connection with Sseveral Machines
This section provides an example of connecting PBVR Server and PBVR Client on two remotemachines ‘machineA’ and ‘machineB’ via ‘machineC’ for some reason, e.g. security. Once the SSH port
forwading is established, the launching method is basically the same as the stand-alone mode, as wtih
the two point remote concection mentioned before.
Step 1 [SSH port forwarding A -> C]machineA> ssh -L 60000:localhost:60000 username@machineC(Forwarding the 60000 port of machineA to the 60000 port of machineC)
Step 2[SSH port forwarding C -> B]machineC> ssh -L 60000:localhost:60000 username@machineB（Forwarding the 60000 port of machineC to the 60000 port of machineB）

４.４. Launching Daemon and Port Forwarding
The daemon is started at an interactive node or job, and performs socket communication with the
PBVR client. The user performs ssh port forwarding from the terminal on your PC to the interactive node,
moves to the directory where the daemon load module is located, and starts the daemon as follows.
$ ./pbvr_daemon
first reading time[ms]:0
Server initialize done
Server bind done
Server listen done
Waiting for connection ...
As described above, when waiting for the socket communication connection with the client, start the
PBVR client from another terminal. The default port number at daemon startup is 60000. The port
number can be changed with the command line option -p at startup as shown below.
$ ./pbvr_daemon –p 71000
The daemon aggregates particle files and updates transfer function files with reference to theenvironment variables. Then, daemon sends and receives data with the PBVR client through the port
specified at startup.

14

５ Particle Sampler
By inserting the generate_particles function into the simulation code, the particle sampler can
generate particles for in-situ visualization. This function is defined in kvs_wrapper.h and can be used byreferencing and linking particle generation libraries.
The particle sampler is executed with batch processing of the simulation, and converts the calculated
volume data to the particle data referring to the environment variables (section ４.１) and the transfer
function file (section ４.２). In the directory specified by PARTICLE_DIR, the particle sampler outputs
a particle data file and a t_pfi_coords_minmax.txt file which has the maximum and minimum coordinatesof the area. The particle data file consists of a header file (kvsml), a coordinate file (coord.dat), a color
file (color.dat), and a normal file (normal.dat), output one set per node at each time step. At the sametime, the particle sampler outputs $TF_NAME_timestep.tf file recording history of transfer functionchange, history_timestep.txt file recording histogram and the range of physical values, state.txt
recording time step interval in VIS_PARAM_DIR.

５.１. Particle Generation Function for Structured Grid#include “kvs_wrapper.h”
void generate_particles(
int time_step, domain_patameters dom, Typs** volume_data, int num_volume_data );This function takes as arguments the time step of simulation, information on calculation area, andvolume data of simulation result.
・

int time_step：simulation time step・

domain_patameters dom：data structure defines following calculation domain
typedef struct{
float x_global_min; // Minimum of x coordinates in whole domain
float y_global_min; // Minimum of y coordinates in whole domain
float z_global_min; // Minimum of z coordinates in whole domain
float x_global_max; // Maximum of x coordinates in whole domain
float y_global_max; // Maximum of ycoordinates in whole domain
float z_global_max; // Maximum of z coordinates in whole domain
float x_min;// Minimum of x coordinates in subdomain

float y_min;// Minimum of y coordinates in subdomain

float z_min;// Minimum of z coordinates in subdomain

int* resolution;// Pointer to int resolution[3]float cell_length;//Length of a cell

} domain_parameters;
・Typs ** volume_data: A pointer to an array of volume data of simulation results. “Type” is a type ofuser-specified physical value, and multi-variable volume data is defined as a two-dimensional array.
15

In volume data of lattice resolution (X, Y, Z), the value of the position (i, j, k) of the n-th variable isreferred to as volume_data [n] [i + j * X + k * X * Y] .
・

int num_volume_data：number of volume data

５.２. Particle Generation Function for Unstructured Grid#include “kvs_wrapper.h”
void generate_particles(
int time_step, domain_parameters dom, Type** values, int nvariables, float* coordinates,
int ncoords, unsigned int* connections, int ncells );This function takes as arguments the time step of the simulation and information on the calculationarea, volume data of the simulation result, and lattice information of the unstructured lattice.
・

int time_step：simulation time step・

domain_patameters dom：data structure defines following calculation domain
typedef struct{
float x_global_min; // Minimum of x coordinates in whole domain
float y_global_min; // Minimum of y coordinates in whole domain
float z_global_min; // Minimum of z coordinates in whole domain
float x_global_max; // Maximum of x coordinates in whole domain
float y_global_max; // Maximum of ycoordinates in whole domain
float z_global_max; // Maximum of z coordinates in whole domain} domain_parameters;

・Typs ** volume_data: A pointer to an array of volume data of simulation results. “Type” is a type ofuser-specified physical value, and multi-variable volume data is defined as a two-dimensional array.
The value on the cell vertex of the nth variable is referred to by volume_data [n] [cell].

・

float* coordinates: Pointer to an array of vertex coordinates. The i-th vertex coordinates (x, y, z) arereferenced by (coordinates [3 * i], coordinates [3 * i + 1], coordinates [3 * i + 2]).

・

int ncoords：number of coordinates・

unsigned int * connections: A pointer to the connection list of vertex IDs that make up a hexahedral
element. The configuration of the hexahedral element is shown in Figure ５-1. The n-th vertex of
the i-th hexahedral element is referred to by connections [8 * i + n].

・

int ncells：number of elements.

・

pbvr::VolumeObjectBase::CellType& celltype : element type. IS-PBVR supports tetrahedral,
hexahedral, pyramid, prism, quadratic tetrahedral, and quadratic hexahedral elements. Each typeis defined in VolumeObjectBase class written in following file.
/DaemonAndSampler/InSituLib/unstruct/TFS/VolumeObjectBase.h

16

0

3
1

2

4

7
5

6

Figure ５-1 Connection of vetices of a hexahedral element

５.３. Particle Generation Function for AMR Grid
In-Situ PBVR supports Block Structured AMR, which is optimized for the memory layout of many-corecomputers. In this type of grid, an orthogonal grid of N3 is defined as a unit of the minimum processing
area called “leaf”, and leaves of different sizes are connected in each layer. Therefore, Block Structured
AMR is defined as a four-dimensional grid of NxNxNxL (L is the number of leaves). Figure ５-2 showsa two-dimensional example.

Figure ５-2

Example of a two-dimensional hierarchical grid. The upper part is the lattice of thehierarchy Lv. 1 and the lower part is the hierarchy Lv. 2. The leaf is defined by a 2 × 2 orthogonal grid,
red is a leaf of Lv.1, blue is a leaf of Lv.2.
#include “kvs_wrapper.h”
void generate_particles(
int time_step, domain_patameters dom,
std::vector<float>& leaf_length,
std::vector<float>& leaf_min_coord,
int nvariables, float** values);
This function takes as arguments the time step of simulation, information of calculation area,
configuration information of hierarchical lattice, and volume data of simulation result.
・

int time_step：simulation time step・

domain_patameters dom：data structure defines following calculation domain

17

typedef struct{
float x_global_min; // Minimum of x coordinates in whole domain
float y_global_min; // Minimum of y coordinates in whole domain
float z_global_min; // Minimum of z coordinates in whole domain
float x_global_max; // Maximum of x coordinates in whole domain
float y_global_max; // Maximum of ycoordinates in whole domain
float z_global_max; // Maximum of z coordinates in whole domain
int* resolution;// Pointer to int resolution[4]} domain_parameters;
・・

std::vector<float>& leaf_length： A reference to an array of leaf lengths. The length of the l-th leaf
is referred to by leaf_length [l].

・

std::vector<float>& leaf_min_coord： A reference to an array of leaf minimum position coordinates.
The coordinates of the l-th leaf are referenced by (leaf_min_coord [3 * l], leaf_min_coord [3 * l + 1],
leaf_min_coord [3 * l + 2]).

・

float** values： Pointer to the array of resulting volume data. Multivariate volume data is defined
as a two-dimensional array. In the volume data of the grid resolution (X, Y, Z, L), the value of theposition (i, j, k, l) of the nth variable is values [n] [i + j * X + k * X * Y Refer to + l * X * Y * Z].

・

int nvariables：number of variables

５.４. Connection into Simulation CodeIn-Situ visualization is possible by compiling the generate_particles function into the simulation codeof the user. This chapter shows the procedure for connecting particle sampler to test simulation code as
an example. The test simulation code calculates the value of charge density on the lattice vertex at eachtime step according to the charge density equation of hydrogen. Therefore, the class name to be
calculated is Hydrogen. The code without particle sampler integration is as follows.
#include "Hydrogen.h"#include <iostream>#include <mpi.h>
int main( int argc, char** argv ){MPI_Init(&argc, &argv);
Hydrogen hydro;
int time_step = 0;
for(;;){hydro.values;
time_step++;}
MPI_Finalize();
return 0 ;}18

In the above source code, Hydrogen class outputs volume data of each time step in hydro.values in forloop.

５.４.１. Domain Information and Particle Generation Function
The

generate_particles

functionobtainsdomaininformation

throughthestructure

domain_parameters. The user needs to copy the area information obtained from the simulation code to
the structure.
int mpi_rank;
MPI_Comm_rank( MPI_COMM_WORLD, &(mpi_rank) );int resol[3] = { hydro.resolution.x(), hydro.resolution.y(), hydro.resolution.z() };domain_parameters dom = {hydro.global_min_coord.x(),hydro.global_min_coord.y(),hydro.global_min_coord.z(),hydro.global_max_coord.x(),hydro.global_max_coord.y(),hydro.global_max_coord.z(),hydro.global_region[mpi_rank].x(),hydro.global_region[mpi_rank].y(),0.0,resol,hydro.cell_length};
In the above code, the rank number of MPI is used to calculate area of subdomain of Hydrogen. The
domain information, resolution, MPI rank, and volume data becomes arguments of generate_particles
function. The generate_particles function is called after simulation inside the time step loop. In the caseof Hydrogen, it is inserted as follows.

int time_step = 0;
for(;;) {generate_particles( time_step, dom, hydro.values, hydro.nvariables )time_step++;}

19

６ PBVR Client
In default mode, PBVR client draws particle data of the latest time step received from daemon. Viewer
of PBVR client draws particle data using OpenGL. The PBVR client provides a GUI to edit visualization
parameters, including transfer functions, and sends the visualization parameters to the daemon.
Sending and receiving data between the daemon and the PBVR client is performed through an arbitrary
port number by socket communication.

６.１. Launching PBVR Client
The following examples show how to launch PBVR client.
$ pbvr_client [command line options]
表 ６-1 Command line options for PBVR client
optionvalue

default

explanation

-p

ポート番号

60000

port number-viewer

100 ～ 9999 ×

620×620

resolution of PBVR client-

shading method ※1

100～9999-shading{L/P/B},ka,kd,ks,n※1. This argument specifies the shading parameters.

L: Lambert ShadingThis method ignores specular reflection in the shading process.
Parameters ‘ka’ and ‘kd’ are the coefficient for ambient and diffusion, respectively.
They can have a value between 0-1.
P：Phong ShadingThis method adds the specular reflection to Lambert shading. Phong shading imitatessmooth metal and mirrors. (This is sometimes called highlight).
Parameter ‘ka’, ’kd’,’ks’ (coefficients for specular reflection lying between 0-1) and ’n’(strength of highlight lying between 0-100) are used.
B：Blinn-Phong ShadingThis is a shading model that simplifies Phong shading. Parameters ‘ka’, ‘kd’, ‘ks’, and‘n’ exist.

６.２. Terminating of PBVR Client
PBVR client is terminated by pressing Ctrl + c on the console where you started the program. Whenyou press Ctrl + c, the PBVR client synchronizes with the daemon at the time of the time update, andboth ends. However, if communication is interrupted with the Stop button on the time step control panel,
the Ctrl + c key input is ignored. Also note that if you terminate the daemon with Ctrl + c, you will not beable to terminate the PBVR client with Ctrl + c.
20

６.２.１. Forced Termination
If the PBVR client and daemon are not terminated by pressing the Ctrl + c key for some reason, forced
termination is required using kill command with the process numbers of the client and daemon by pscommand as shown below.
【Forced termination of PBVR client】$ ps -C PBVRViewer
PID TTY
19582 pts/6

TIME CMD
00:00:00 PBVRViewer$ kill -9 19582【Forced termination of daemon】$ ps -C CPUServer
PID TTY
19539 pts/5

TIME CMD
00:00:00 CPUServer$ kill -9 19539

21

６.３. Using PBVR Client GUI
The PBVR client shows the visualization result and provides the GUI that can interactively control thevisualization parameters.

６.３.１. Viewer
As shown in Figure６-1, Viewer renders the rendering result of particle data.

Figure６-1 Viewer[Operations]Rotation:

Dragging left with the mouse

Translation:

Dragging right with the mouse

Zoom:

Shift + left-dragging, or dragging while pressing the mouse wheel

Reset:

Home button (fn + left arrow on Macs)

22

[Display]time step:Time step for the data displayed

fps:

Frame rate (frame/sec)[Tool bar]
Zoom in/out.

Translations.
Get

keyframe/delete

animation.

23

keyframe/play

keyframe

６.３.２. Tool Bar
Various functions of the client program are selected from the toolbar. The “File” tab controls the input
and output of the visualization parameter file and transfer function.
The visualization parameter means a group of parameters that can be set by the client, such as viewer
resolution, particle density, particle number limit, input volume data file name on the server (PFI file or
PFL file), transfer function, etc. The visualization parameter file is a file in which they are described in
the tag format.
The "File" tab and its functions are shown below.

Figure ６-2 File tab•

Import Parameters specifies the visualization parameter file to be input.

•

Export Parameters outputs the visualization parameter file.

•

Import Transfer Functions specifies the input transfer function file.

•

Export Transfer Functions outputs a transfer function file.

The "Window" tab and its functions are shown below.

Figure ６-3 Window tab•

Animation Control displays a video creation panel.

•

Coordinates displays a coordinates editor.

•

Filter Information displays the input volume data property.

•

Legend Option displays a legend panel for control the legend bar.

•

System Status displays system property.

•

Render Options controls rendering options.

•Time Controls displays a time step control panel.

24•Volume Transform specifies geometry transformation for a rendering object.

•

Revert to Default Layout returns the layout of GUI when this application starts.

•

Particle Selector displays a particle panel.

•

Transfer Function Editor displays a transfer function editor.

•

Viewer Controls displays a viewer control panel.

25

６.３.２.１ FilterInfo
The Filter Info panel displays information about the input filtered volume data.

Figure ６-4 FilterInfo•

Vector number displays the number of variables consisting of the volume data.

•

All Elements displays the number of elements consisting of the volume data.

•

Sub Volume displays the number of sub-volume separated by the filter program.

•Element Type displays the type of element.

•

All Nodes displays the number of nodes of the volume data.

•

Step Number displays the number of time steps.

•

File Type displays the volume decomposition type of a filter program used in client-severtype PBVR. It is unused in this in-situ PBVR.
•

3D Extent displays the min-max X-Y-Z coordinates of the volume data.

26

６.３.２.２ System Status

Figure ６-5 System status•

CPU Memory displays the system memory usage in megabytes.

•

Displayed Particles displays the number of rendering particles.

•

Step Interval specifies minimum of a time step update interval by [msec]. This is used tolengthen the update interval for time steps that are too short.

27

６.３.２.３ Render Options

Figure ６-6 Render Options
Density specifies the particle density related to the depth of the image.
Limit specifies the upper limit of the number of particles generated by the server program in order toprevent the number of particles from exploding due to incorrect transfer function specification. If thenumber of particles exceeds this upper limit, the server program automatically reduces the image quality
and adjusts so that the number of particles falls within this upper limit.
Data-Size Limit specifies the upper limit of the particle data size using [GB] generated by the server
program in order to avoid explosion of the number of particles due to incorrect transfer function
specification. If the particle data size exceeds this upper limit, particle generation is forced to stop.
Resolution specifies the viewer’s resolution.

28

６.３.２.４ Volume Transform

Figure ６-7 Volume Transform•

Rotation specifies the rotation angle (degree) about the x-, y-, and z-axis of the object.

•

Translation specifies the translation of the object in the x, y and z directions.

•

Scale specifies the scale rate of the object.

29

６.３.３. Transfer Function Editor
The transfer function editor enables to create transfer functions which assign the colors and opacity tophysical values. In a standard rendering of a volume, a transfer function is defined with only one physicalquantity. In contrast, PBVR allows for multi-dimensional transfer functions. It has the following three
features:1)

It assigns a parameter to color and, independently, a parameter to opacity.

2)

It defines both color and opacity with an arbitrary function of the X-, Y-, and Z-coordinatesand parameters q1, q2, q3, ….

3)It algebraically synthesizes a multi-dimensional transfer function from one-dimensionaltransfer functions that are defined by color functions C1, C2, … and opacity functions O1,
O2, ….

Figure ６-8 Transfer Function Editor[Operations]
Scale change in histogram: Drag the mouse up/down in the histogram.

30

•Number Transfer Functions specifies the maximum number of transfer functions thatcan be created.
•

Color Function category specifies the color functions and its argument which issynthesized by the physical values.
•

Opacity Function category specifies the opacity functions and its argument which issynthesized by the physical values.
•

Color Map category edits the color functions.

•

Opacity Map category edits the opacity functions.

•

Reset button returns the transfer functions to initial state.

•

Apply button sends the transfer function to the server program.

•

Export button saves the transfer function file with same format of “-pa” option.

•

Import button reads the transfer function file.

[Available operatos for algebraic equations]
Binary operators +, -, *, /, ^Functiobns sqrt(),cbrt(), log(), log10(), exp(), abs(), floor(), ceil(), sin(), cos(), tan, asin(),sinh(), cosh(), tanh(), asinh(), acosh(), atanh(), heavi(), rectfunc(),
Binary functions sigmoid(*,*), gauss(*,*)

31

６.３.３.１ Color Map Specification

Figure ６-9 Top: Color Function category, bottom: Color Map category[Color Function category]•

Synthesizer specifies the synthesis of the color function of C1 to C [N]. *1•

Function spin button selects the color function of C1 to C [N] for edit its argument.

•

The button surrounded by red dotted line opens Color Function Editor which enables tosynthesize the physical values as an argument of C1 to C [N].

[Color Map category]•User Defined Range Min:Max specifies the minimum and maximum range of the color

function which is assigned to the synthesized physical value.
•

Synth. Func. Range Min:Max displays the minimum and maximum values of thesynthesized physical value.
•

Edit Color Map button opens Color Map Editor.

•

Histogram displays a distribution of the synthesized physical value with the user defined

range min:max.
*1 [N] is the value of the limit number of the transfer function specified by Number of Transfer Functions.

32

６.３.３.１.１. Color Function Editor
PBVR Client can define the arguments of the color functions C1 to C [N] using the algebraic formula. In
the Color Function Editor, the algebraic formula is synthesized by the physical values. The names of thevariables that can be used in the algebraic formula are shown below.

•

Physical values：q1, q2, q3, .., qn.

•

Coordinate values：X, Y, Z.

Figure ６-10 Color Function Editor•

Color Function List displays the created color functions.

•

Function: C[N] = f( algebraic formula ) displays a color function selected in Color

Function List and enables to edit the algebraic formula.
•

Cancel button close this panel.

•

Set button applies the edited algebraic formula to Color Function List.

•

Save button applies the Color Function List to the transfer function•

Select button applies a color function of Color Function List to Function.

33

６.３.３.１.２. Color Function Editor: Freeform Curve Edit Tab
In the Freeform Curve Edit tab of Color Map Editor, the color function can be created by the free form
curve.

Figure ６-11 Color Map Editor: freeform curve tab•

Color Palette consists of saturation on the horizontal axis and brightness on the verticalaxis, and these can be specified by the position of the cursor.
•

The right side of the color palette is RGB bar and enables to specify the hue. Color Palettereflects the hue specified in RGB bar.

•

Reset button returns this tab to default state.
34

•Undo button undo single action.

•

Redo button redo single action.

•

Save button applies created color function.

•

Cancel button close this panel.

Blends the single color specified by the Color palette and the RGB bar with those from a standard
spectrum. This is done by dragging the mouse along the color box while pressing the left mouse button.
The blending ratio between the specified color and the color from the standard spectrum is determinedby the vertical position of the mouse within the box. For example, if the mouse is traced across the upperedge of the color box from left to right, it is painted completely by the specified color. If the mouse istraced across the middle of the color box, the colors in the box are replaced with blended colors that are
50% of the original color and 50% of the specified color.

35

６.３.３.１.３. Color Map Editor: Expression Tab
In Expression Edit tab of Color Map Editor, the color functions can be created by algebraic expressions.

Figure ６-12 Color Map Editor: Expression tab•

R describes the color function of the red component of the color by algebraic expression.

•G describes the color function of the green component of the color by algebraic

expression.
•

B describes the color function of the blue component of the color by algebraic expression.

A variable of the color function is x, and the domain and range of the color function are 0 to 1.

36

６.３.３.１.４. Color Map Editor: Control Points Edit Tab
In Control Points Edit tab of Color Map Editor, the color functions can be created by control points.

Figure ６-13 Color Map Editor: Control Points Edit Tab•

Point enables to specify the values of control points (up to 10). The domain is 0 to 1.

•

Red enables to specify a red component value corresponding to the control point. The range

is 0 to 1.
•

Green enables to specify a green component value corresponding to the control point. The

range is 0 to 1.
•

Blue enables to specify a blue component value corresponding to the control point. The

range is 0 to 1.
Each control point is interpolated with a piecewise linear function.

37

６.３.３.１.５. Color Map Editor: Predefined ColorMaps Tab
Predefined Color Maps tab of Color Map Editor provides the selection of predefined color functions.

Figure ６-14 Color Map Editor: Predefined ColorMaps Tab

Default Colors: displays the list of the predefined color maps. The following templates are available.

•

Rainbow•

Blue-white-red•

Black-red-yellow-white•

Black-blue-violet--yellow-white•

Black-yellow-white38•Blue-green-red•

Green-red-violet•

Green- blue--white•

HSV model•

Gray-scale

•

Black

•

White

39

６.３.３.２ Opacity Map Specification

Figure ６-15 Top: Opacity Function category, bottom: Opacity Map category[Opacity Function category]•

Synthesizer specifies the synthesis of the opacity function of O1 to O[N]. *1•

Function spin button selects the opacity function of O1 to O[N] for edit its argument.

•

The button surrounded by red dotted line opens Opacity Function Editor which enables
to synthesize the physical values as an argument of O1 to O[N].

[Opacity Map category]•

Range Min:Max specifies the minimum and maximum range of O1 to O[N].

•Server Side Range (min:max) displays the minimum and maximum values of thesynthesized physical value•

Edit Color Map button opens Opacity Map Editor.

*1 [N] is the value of the limit number of the transfer function specified by Number of Transfer Functions.

40

６.３.３.２.１. Opacity Function Editor
PBVR Client can define the arguments of the opacity functions O1 to O [N] using the algebraic formula.
In the Opacity Function Editor, the algebraic formula is synthesized by the physical values. The namesof the variables that can be used in the algebraic formula are shown below.

•

Physical values：q1, q2, q3, .., qn.

•

Coordinate values：X, Y, Z.

Figure ６-16 Opacity Function Editor•

Opacity Function List displays the created color functions.

•

Function: O[N] = f( algebraic formula ) displays a color function selected in Opacity

Function List and enables to edit the algebraic formula.
•

Cancel button close this panel.

•

Set button applies the edited algebraic formula to Opacity Function List.

•

Save button applies the Opacity Function List to the transfer function•

Select button applies a color function of Opacity Function List to Function.

41

６.３.３.２.２. Opacity Map Editor: Freeform Curve Tab
In the Freeform Curve Edit tab of Opacity Map Editor, the opacity function can be created by the freeform curve.

Figure ６-17 Opacity Map Editor: Freeform Curve Tab•

Reset button returns this tab to default state.

•

Undo button undo single action.

•

Redo button redo single action.

•

Save button applies created color function.

•

Cancel button close this panel.

In drawing the opacity function, a free-form curve can be drawn by left-dragging with the mouse and alinear interpolation line between two points can be drawn by right-click.

42

６.３.３.２.３. Opacity Map Editor: Opacity Expression Tab
In Expression Edit tab of Opacity Map Editor, the opacity functions can be created by algebraic
expressions.

Figure ６-18 Opacity Map Editor: Opacity Expression Tab•

O: describes the color function of the blue component of the color by algebraic expression.

A variable of the opacity function is x, and the domain and range of the opacity function are 0 to 1.

43

６.３.３.２.４. Opacity Map Editor: Control Point Editor Tab
In Control Points Edit tab of Opacity Map Editor, the opacity functions can be created by control points.

Figure ６-19 Opacity Map Editor: Control Point Editor Tab•

Point enables to specify the values of control points (up to 10). The domain is 0 to 1.

•

Opacity enables to specify the opacity value corresponding to the control point. The range

is 0 to 1.
Each control point is interpolated with a piecewise linear function.

44

６.３.３.３ Function Editor
Table ６-1 lists the built-in math operations available in the function editors. They can be used for theoverall transfer functions, the parameters, and for the color and opacity curves.
Table ６-1 Math operations available in function editors
Math operation

In function editors

+

+

-

-

×

*

/

/

sin

sin(x)cos

cos(x)tan

tan(x)log

log(x)exp

exp(x)

square rootsqrt(x)

power

x^y

When arithmetic processing by the function editor produces NaN, PBVR outputs an error message andstops the drawing process.

45

６.３.４. Time panel
Figure６-20 shows Time panel, which specifies the time steps for the visualization. Each widget works
as described below.

Figure６-20 Time panel•

Time step specifies the time step to be rendered.

•

If Last step check box is checked, the viewer displays the latest time step.

•

Start/Stop starts/stops communications between PBVR Client and PBVR Server.

46

６.３.５. Particle and Polygon composition
CS-PBVR supports composition of the volume rendering and polygon rendering. Figure６-21 shows
Particle panel, which is used to integrate multiple particle datasets. Each widget works as described
below.

Figure６-21 Particle panel. Left: particle selection mode, right: polygon selection mode.
Display Particle category shows a list of particle and polygon datasets. The particle is sent from PBVR
Server, or is loaded from local files (maximum 5 files). The polygon is loaded from local files (maximum
5 files).
Particle panel supports STL format polygon data. This function can process STL files that are divided
into one file per time step according to the following naming convention.
prefix_*****.stl (***** means the number of time steps in a five-digit display.)If the subpixel levels of the server particle data and local particle data is different, the server particle’s
subpixel level takes priority. In standalone mode, If multiple local particle data are to be displayed andthe sub-pixel level of each particle data is different, the sub-pixel level of the later loaded particle datatakes priority.
Keep Initial Step category specifies particle/polygon datasets, in which the initial step data isdisplayed before the time series starts, when integrated particle/polygon datasets start from different
time steps. Keep Final Step category specifies particle/polygon datasets, in which the final step data isdisplayed after the time series ends, when integrated particle/polygon datasets end at different time
steps.

47

[Display Particle/ Keep Initial Step/ Keep Final Step category]•Server check box is activated when a particle dataset from PBVR Server is integrated

with local particle data sets. This checkbox is not available in stand-alone mode.
•(Particle1)- (Particle5) check boxes are activated to integrate the particle datasets loaded

from local files. The checkbox is not available before particle datasets are loaded via
Particle file panel.
•(Polygon1)- (Polygon5) check boxes are activated to integrate the polygon datasets loaded

from local files. The checkbox is not available before polygon datasets are loaded via
Particle file panel.
Edit Particle category enables to add particle data or polygon from a local file on the client PC to thelist of (Particle1)- (Particle5), (Polygon1)- (Polygon5) and to save the integrated particle data as theparticle file.
[Edit Particle category]•

Item spin button selects the item from the list of (Particle1)- (Particle5), (Polygon1)- (Polygon5) to
add particle data.

•

Particle Name text box enables to name the particle data and polygon read from the client PC. If
this description is omitted, the file name displayed in the Particle File column is used.

•

Browse button opens the file dialogue to load the particle data and polygon. The path of the loaded
particle data and polygon file is displayed in the Particle File column. A path containing a doublebyte character string cannot be specified.

•

Add button adds the particle data and polygon displayed in the Particle File column to the item
selected with the Item spin button. If you select an item that has already been added, overwrite itwith the particle data that was load later.

•

Delete button deletes the particle data and polygon added to the item currently selected with theItem spin button.

•

Export button integrates the particle data in memory and outputs it to the file specified in the Particle
File column.

•

Close button closes Particle panel.

•

Polygon Option appears in case of that polygon is selected at Item spin button. It can specify colorand opacity of the polygon.

６.３.５.１ Example of volume and polygon composition
The following is an example of the composition of the test program Hydrogen which is included in
Example directory and the polygon hydrogen.stl of its boundary shape. After portforwarding client andserver, the test program Hydrogen and the daemon is launched. Next, the volume rendering on the clientis executed (Figure ６-22 left). Next, in the Edit Particle category of the Particle Integration panel, select48

Polygon from the Item spin box, and specify the path of hydrogen.stl in Particle File. Next, specify thedata name "hydrogen.stl" in Particle Name, and set the color and opacity. Then, check hydrogen.stl in
the Display Particle category (right side of Figure ６-22), and the boundary shape will be synthesized(center of Figure ６-22).

Figure ６-22 Left: volume rendering result of Hydrogen; Center: composite view of
Hydrogen volume and its boundary shape; Right: settings of the Particle Integration panelin the composite view (Replace spx with hydrogen in the above figure.).

６.３.５.２ Example of multi-time steps composition
The behavior of the particle integration is explained using Figure ６-23 (sever side: 1~4 time steps,
client side:0~3). If the server checkbox and the particle checkboxes are checked, all time steps are
displayed as shown in Table ６-2. If the checkbox Keep Initial Step is checked for the client particles,
only the first time step is displayed, as shown in Table ６-3. If Keep Final Step is checked for the clientparticles, only the final time step is displayed, as shown in Table ６-4.

S1

ServerS2

S3

S4
time steper
Client

C0

C1

C2

C3

er
Figure ６-23 The behavior of the particle integration

Table ６-2 Particle datasets displayed by default.
Step0

Step1

Step2
49

Step3

Step4

Server

-

S1

S2

S3

S4

Client

C0

C1

C2

C3

-

Table ６-3 Particle datasets displayed with Keep Initial Step for the server particles.
Step0

Step1

Step2

Step3

Step4

Server

S1

S1

S2

S3

S4

Client

C0

C1

C2

C3

-

Table ６-4 Particle datasets displayed with Keep Final Step for the client particles.
Step0

Step1

Step2

Step3

Step4

Server

-

S1

S2

S3

S4

Client

C0

C1

C2

C3

C3

50

６.３.６. Image file production
In Animation Controls, PBVR Client saves image data for the Viewer in two modes, both of which areplayed as a movie.
lTime series data mode
The images are saved as time series data in BMP format. The image data files are converted
and compressed into a movie file via free software, such as ImageMagic and ffmpeg.

l

Key frame animation mode
Geometrical information from an arbitrary time step is retained as a key frame. A series of key
frames can be played as a key frame animation.

Figure６-24 shows the Animation Control Panel. Each widget works as described below.

Figure６-24 Animation Controls Panel•

Capture spin box turns image production off or on.

•

Image File specifies a prefix for the image data files. The default name is “PBVR_image”.

•

File specifies a key frame file that contains geometrical data. The default nameis ./xform.dat.
•

Interpolation specifies the number of frames used for linear interpolation of the geometry

data between two key frames in a key frame animation. The default value is 10.
•

Total Key Frames shows the number of key frames stored in the current key frame

animation. It is initialized to 0, and incremented (or decremented) by pressing “x” (or “d”).
It is initialized to 0 again by pressing “D”.

51

•Total Animation Frames shows the number of total frames stored in the current key

frame animation, which is calculated as(Total Key Frames – 1) x Interpolation

６.３.６.１ Image production
Image files are produced as follows.
1.

Specify the prefix for the image files in image file.

2.

Select “on” in the capture drop down menu.

3.

A series of image files are saved at each time step.

4.

Image production is stopped by selecting “off” in the capture drop down menu.

The image files are saved in the directory specified by the command line option “-iout”. If “-iout” option
is not specified, they are saved in the current directory “./”. The following shows an example of image
data files produced with the default prefix “PBVR_image”:
PBVR_image.00001.bmp
PBVR_image.00002.bmp…
If the image files are produced from a key frame animation, which is explained later, the file names are
modified by adding “_k” after the prefix.
PBVR_image_k.00001.bmp
PBVR_image_k.00002.bmp…

６.３.６.２ Key frame animation of a still image
A key frame animation of a still image, which is obtained by pressing Stop in the Time Panel, is
produced as follows.
[Capture key frames and save them in a file]1) Specify a key frame file in file.
2) Activate the Viewer by clicking it.
3) Adjust the view and press “x” to store the geometry information for the view inmemory.
4) Repeat (3).
5) Press “M” (i.e., shift+m) to play the key frame animation.
6) If the contents of the key frame animation are satisfactory, press “S” (i.e., shift+s)
52

to save the geometry information as a series in the key frame file.
[Play a key frame file]1) Specify a key frame file in file.
2) Activate the Viewer by clicking it.
3) Press “L” (i.e., shift+f) to play the key frame animation stored in the key frame

file.
4) Press “x” to add new key frames to the current key frame animation.

Table６-5 Keys used for controlling key frame animation
Key

Functionx

Add geometry information for the current view to the key frame
data in memory.

dDelete the last key frame.

D

Delete all key frames.

M

Play or pause the key frame data in memory.

S

Save the key frame data in memory to a key frame file.

L

Load a key frame file and play its key frame data53

６.３.６.３ Key frame animation of time series data
A key frame animation of time series data can be produced as follows.
1)

By pressing “x” while time series data are being rendered, both geometry information and a time
step number are stored in memory.

2)

Press “S” to save a series with geometry information and time step numbers in the key frame
file.

3)

Press “F” to load a series with geometry information and time step numbers in the key frame fileand play a key frame animation. Here, if key frames are at unequal intervals, then interpolation
frames, which are specified in interpolation, are non-uniform in time.
Time series data

Key frame information
No.

Time step number

0

00002

1

00004

2

00025

3

00035

00002,00003
00004,00006,00008～00024
00025,00026,00027～00034
00035～00044

Figure６-25 Key frame animation for time series data

In the example shown in Figure６-25, if there are 10 interpolation frames between key frames, then
5 interpolation frames are assigned to the time steps 00002 and 00003 between key frames 0 and 1.
On the other hand, between key frames 1 and 2, 10 interpolation frames are assigned to the time
steps from 00004 to 00024. As a result, the time steps 00004, 00006, …, 00024 are shown in the key
frame animation.

54

６.３.６.４ Key frame file format
A key frame file contains binary data with the following format.

Type

Size

data

(bytes)int

4Time step number

float

4

rotation[0].x

float

4

rotation[0].y

float

4

rotation[0].zFile format

float

4

rotation[1].x

Key frame data 1

float

4

rotation[1].yKey frame data 2

float

4

rotation[1].z

float

4

rotation[2].x

float

4

rotation[2].y

float

4

rotation[2].z

float

4

translation.x

float

4

translation.y

float

4

translation.z

float

4

scaling.x

float

4

scaling.y

float

4

scaling.z:

Figure６-26 Key frame file format

55

６.３.７. Legend panel
Figure６-27 shows the Legend panel. A legend is a bar showing how the values of a physical quantity
are rendered as a color in the visualization. Each widget works as described below.

Figure６-27 Legend panel•

Display Legend checkbox turns on or off of showing the legend.

•

Caption text box enables to enter a caption string for the legend and the contents arereflected by Set button.
•

Color Function spin button selects the color map legend and the range.

•

Layout Direction spin button selects the direction of the legend (vertical/horizontal).

•Intervals specifies the number of tick marks.

•

Thickness in Division category specifies a thickness of the tick marks.

•

Color in in Division category specifies a color of the tick marks.

•

Thickness in Frameline category specifies the thickness of the frame border.

•

Color in in Frameline category specifies a color of frame border.

Figure６-28 shows an example of legend.

56

Figure６-28 Example of a legend57６.３.８. Viewer Control Panel
Figure６-29 shows the Viewer Control Panel, which specifies the properties of the viewer. Eachwidget works as described below.

Figure６-29 Viewer control panel

・ Background specifies the background color of the viewer.
・ Font selects a font type and size of the character shown to a viewer

58

７ How to Execute Examples
This section shows an example of port-forward connection from Linux/Mac to FUGAKU using
Examples included in the source code package to execute In-Situ PBVR. In this section, we will show
you how to use a login node that are allowed to execute programs and how to use interactive jobs. Thissection also shows an example of accessing a remote server from Windows and executing In-Situ
PBVR.

７.１. Login node
The following is the procedure for running the test program for the unstructured lattice(is_pbvr/Example/Hydrogen_unstruct) on the FUGAKU by using the login node that are allowed to
execute programs. In this example, we assume that the RSA public key and the ssh host name and username have been configured, the source package has been placed in /home/ on FUGAKU, and the build
is complete. In this procedure, the test program combined with In-Situ PBVR is submitted to job, andthe daemon is launched on the login node and the PBVR client is launched on the user PC.
Start up the first terminal, log in to FUGAKU, and submit the test program to the job using a script"run.sh".
［Terminal 1］[userPC]$ ssh UseerID@login.fugaku.r-ccs.riken.jp[fugaku]$ cd /home/is_pbvr/Example/Hydrogen_unstruct[fugaku]$ mkdir particle_out[fugaku]$ cat run.sh#!/bin/sh#PJM -L "node=1"#PJM -L "rscunit=rscunit_ft01"#PJM -L "rscgrp=small"#PJM -L "elapse=0:60"#PJM --mpi max-proc-per-node=4,proc=4#PJM -sexport OMP_NUM_THREADS=12
export TF_NAME=default
export VIS_PARAM_DIR=$PJM_JOBDIR
export PARTICLE_DIR=$PJM_JOBDIR/particle_out
mpiexec -n 4 ./run qsub run.sh[fugaku]$ pjsub run.sh59

In the second terminal, make a port-forward connection between FUGAKU and the user PC and startthe daemon.
［Terminal 2］[userPC]$ ssh -L 61000:localhost:61000 UserID@loginX.fugaku.r-ccs.riken.jp[fugaku]$ source /opt/intel/bin/compilervars.sh intel64[fugaku]$ cd /home/is_pbvr/Daemon[fugaku]$ export VIS_PARAM_DIR=/home/is_pbvr/Example/Hydrogen_unstruct

X は 1~6
Activation ofIntel compiler[fugaku]$ export PARTICLE_DIR=$VIS_PARAM_DIR/particle_out[fugaku]$ ./pbvr_daemon -p 61000
In the third terminal, launche the PBVR client on the user PC.
［Terminal 3］[usrPC]$ ./pbvr_client -p 61000

７.２. Interactive Job
Start up the first terminal, log in to Fugaku, and submit the test program to the job using a script"run.sh".
[Terminal 1][userPC]$ ssh FUGAKU[fugaku]$ cd /home/is_pbvr/Example/Hydrogen_unstruct[fugaku]$ mkdir particle_out[fugaku]$ cat run.sh#!/bin/sh#PJM -L "node=1"#PJM -L "rscunit=rscunit_ft01"#PJM -L "rscgrp=small"#PJM -L "elapse=0:60"#PJM --mpi max-proc-per-node=4,proc=4#PJM -sexport OMP_NUM_THREADS=12
export TF_NAME=default
export VIS_PARAM_DIR=$PJM_JOBDIR
export PARTICLE_DIR=$PJM_JOBDIR/particle_out
mpiexec -n 4 ./run qsub run.sh

60

Since Fugaku is not allowed to execute programs on the login node, the computation node and theuser PC are connected port-forward and the daemon is executed in an interactive job.
Start the interactive job on terminal 2 and get the IP address of the computation node running theinteractive job. “ip” command displays the network information, and a IP address after “inet” is thecomputation node’s IP.
[Terminal2][userPC]$ ssh FUGAKU[fugaku]$ cd /home/is_pbvr/Example/Hydrogen_unstruct[fugaku]$ ln -s /home/is_pbvr/Daemon/pbvr_daemon[fugaku]$ pjsub --interact -L "node=1" -L "rscgrp=int" \[fugaku]$ -L "elapse=0:10:00" --sparam "wait-time=600"[intrct]$ ip address show tofu1……
inet **.**.**.**/12 brd 10.255.255.255 scope global noprefixroute tofu1……
Using the IP address of the computation node, port forward connection is made between the user PC
and the computation node at terminal 3.
[Terminal3][userPC]$ ssh -L 60000: **.**.**.**:60000 FUGAKURun the daemon at terminal 2.
[Terminal2][intrct]$ export TF_NAME=default[intrct]$ export VIS_PARAM_DIR=$PJM_JOBDIR[intrct]$ export PARTICLE_DIR=$PJM_JOBDIR/particle_out[intrct]$ ./pbvr_daemon
Launch PBVR client on the user PC in terminal 4.
[userPC]$ ./pbvr_client -p 60000

61

７.３. Windows
The procedure for connecting remotely from a Windows client to a remote server is shown below. The
following procedure assumes transfer from the port 60000 of the client to the port 60000 of the server.
To connect PBVR Client in a Windows machine to PBVR Server in a remote machine, setup port
forwarding with the help of an SSH client software such as TeraTerm or Putty. The following shows anexample for TeraTerm.

1)

Launch TeraTerm and hit cancel in the “New connection” dialog.

Figure ７-1

2)

Tera Term dialog 1)

Select Setup > SSH Transfer from the menu bar. Click Add… in the Forwarding Setup
dialog.

Figure７-2

Tera Term dialog 2)623)

In the Select Direction for Forwarded Port dialog, select Forward Local Port andenter the port number to be used for PBVR Client. In the to remote machine text field,
enter the domain name or the IP address of the server. In the port field, enter the port
number to be used on PBVR Server. Click on OK to complete the setup of port forwarding.

Figure７-3

4)Tera Term dialog 3)

Connect to the server. Select File > New Connection from the menu bar. In the New
Connection panel, enter the host name of the serve and click on OK. In the SSH
Authentication panel, enter the user name and passphrase, or specify the location of
the private key file, and click on OK.

Figure７-4

Tera Term dialog 4)
63

The following procedures show how to launch PBVR Server and Client after establishing port
forwarding. This example uses the Visual Studio 2013 x64 Cross Tools command prompt in Visual
Studio 2013 as the terminal for launching PBVR Client.
Step1 [Launch PBVR Server]
Server> mpiexec –n 4 pbvr_server –p port_number
Step2 [Set a client parameter for Windows]
Windows> set TIMER_EVENT_INTERVAL=1000
Step3 [Launch PBVR Client]
Windows> pbvr_client.exe –vin filename –p port_number
Note that PBVR Client on a Windows machine can be launched also by executing a batch file withthe following lines.
set TIMER_EVENT_INTERVAL=1000
pbvr_client.exe –vin filename –p port_number

64