Preface Comment from Ilmi Yoon:

Just one curiosity, component is in much larger granularity compared

to object in terms of reuasability or usage itself. Component is kind of

package of objects that has interfaces to communicate with other
components.

So, components are much more portable and easily re-usably without
knowing

the programming environment of diffenrent components - they can be
different

programming langauges, etc as long as they know the interfaces to each

other\... Just I feel some discussions are related to object-oriented
not,

component-oriented. Maybeit is from my ignorance and/or lacking of
certain

backgrounds from the last meeting.

> =============The Survey=========================
>
> 1\) Data Structures/Representations/Management==================
>
> The center of every successful modular visualization architecture has
> been a flexible core set of data structures for representing data that
> is important to the targeted application domain. Before we can begin
> working on algorithms, we must come to some agreement on common
> methods (either data structures or accessors/method calls) for
> exchanging data between components of our vis framework.
>
> There are two potentially disparate motivations for defining the data
> representation requirements. In the coarse-grained case, we need to
> define standards for exchanging data between components in this
> framework (interoperability). In the fined-grained case, we want to
> define some canonical data structures that can be used within a
> component \-- one developed specifically for this framework. These two
> use-cases may drive different set of requirements and implementation
> issues.
>
> \* Do you feel both of these use cases are equally important or should
> we focus exclusively on one or the other?

Randy: I think that interoperability (both in terms of data and perhaps
more

critically operation/interaction) is more critical than fine-grained

data sharing. My motivation: there is no way that DiVA will be able to

meet all needs initially and in many cases, it may be fine for data to

go \"opaque\" to the framework once inside a \"limb\" in the framework
(e.g.

VTK could be a limb). This allows the framework to be easily populated

with a lot of solid code bases and shifts the initial focus on important

interactions (perhaps domain centric). Over time, I see the fine-grain

stuff coming up, but perhaps proposed by the \"limbs\" rather than the

framework. I do feel that the coarse level must take into account

distributed processing however\...

I want to facilitate interfaces

between packages, opting for (possibly specific) data models that map

to the application at hand. I could use some generic mechanisms

provided by DiVA to reduce the amount of code I need or bootstrap

more rapid prototyping, but it is not key that the data model be

burned fully into the Framework. I certainly feel that the Framework

should be able to support more than one data model (since we have

repeatedly illustrated that all \"realizable\" models have design

boundaries that we will eventually hit.

Pat: I think both cases are important, but agreeing upon the
fine-grained access

will be harder.

John C: Too soon to tell. Focus on both until the issues become more
clear.

Jim: I think for now we need to exclusively focus on exchanging data
between

components, rather than any fine-grained generalized data objects\...

The first order entry into any component development is to \"wrap up

what ya got\". The \"rip things apart\" phase comes after you can glue

all the coarse-grained piece together reliably\...

Ilmi: I think we need to decide the coarse-grain something like SOAP
that wraps

the internal data with XML format. But I think we don\'t need to decide
the

fined-grain since each component can have choose their own way/format
and

then post format to public, so the party who want to use the component
needs

to follow the interface. But if we like to decide initial sets of format

that must/may be supported by diva components, then we can list most
popular

format and choose some/all of them.

JohnS: While I am very interested in design patterns, data structures,
and services that could make the design of the interior of
parallel/distributed components easier, it is clear that the interfaces
between components are the central focus of this project. So the
definition of inter-component data exchanges is preeminent.

Wes: Both are important. The strongest case, IMO, for the
intra-component DS/DM

is that I have a stable set of data modeling/mgt tools that I can use
for

families of components. Having a solid DS/DM base will free me to focus

on vis and rendering algorithms, which is how I want to spend my time.

The strongest case for the inter-component DS/DM is the \"strong
typing\"

property that makes AVS and apps of its ilk work so well.

The \"elephant in the living room\" is that there is no silver bullet.

I favor an approach that is, by design, incremental. What I mean is that

we can deal with structure grids, unstructured grids, geom and other

renderable data, etc. in a more or less piecemeal fashion with an eye

towards component level interoperability in the long term. In the
beginning,

there won\'t be 100% interoperability as if, for example, all data
models

and types were stuffed into a vector bundles interface. OTOH, a more

conciliatory approach will permit forward progress among multiple

independent groups who are all eyeing \"interoperability\". This is the

real goal, not a \"single true data model.\"

-   Do you feel the requirements for each of these use-cases are aligned
    or will they involve two separate development tracks? For instance,
    using \"accessors\" (method calls that provide abstract access to
    essentially opaque data structures) will likely work fine for the
    coarse-grained data exchanges between components, but will lead to
    inefficiencies if used to implement algorithms within a particular
    component.

-   As you answer the \"implementation and requirements\" questions
    below, please try to identify where coarse-grained and fine-grained
    use cases will affect the implementation requirements.

Randy: I think you hit the nail on the head. Where necessary, I see
sub-portions

of the framework working out the necessary fine-grained, efficient,

\"aware\" interactions and datastuctures as needed. I strongly doubt we

would get that part right initially and think it would lead to some of

the same constraints that are forcing us to re-invent frameworks right

now. IMHO: the fine-grain stuff must be flexible and dynamic over

time as development and research progress.

Pat: I think the focus should be on interfaces rather than data
structures. I

would advocate this approach not just because it\'s the standard

\"object-oriented\" way, but because it\'s the one we followed with FEL,

and now FM, and it has been a big win for us. It\'s a significant
benefit

not having to maintain different versions of the same visualization

technique, each dedicated to a different method for producing the

data (i.e., different data structures). So, for example, we use the same

visualization code in both in-core and out-of-core cases. Assuming up

front that an interface-based approach would be too slow is, in my

humble opinion, classic premature optimization.

Jim: Two separate development tracks. Definitely. There are different
driving

design forces and they can be developed (somewhat) independently (I
hope).

Lori: The TSTT center is not interested in defining a data
representation

per se - that is dictating what the data structure will look like.
Rather,

we are interested in defining how data can be accessed in a uniform

way from a wide variety of different data structures (for both
structured

and unstructured meshes). This came about because we recognize

that

1\. there are a lot of different meshing/data frameworks out there,

that have many man years of effort behind their development,

that are not going to change their data structures very easily

(if at all). Moreover, these infrastructures have made their

choices for a reason - if there was a one-size-fits-all answer,

someone probably would have found it by now :-)

2\. Because of the difference in data structures - it has been very

difficult for application scientists (and tool builders) to experiment

with and/or support different data infrastructures which has

severely limited their ability to play with different meshing
strategies,

discretization schemes, etc.

We are trying to address this latter point - by developing common

interfaces for a variety of infrastructures applications can easily

experiment with different techniques and supporting tool developers

(such as mesh quality improvement and front tracking codes) and

write their tools to a single API and automatically support multiple

infrastructures.

We are also experimenting with the language interoperability tools

provided by the Babel team at LLNL and have ongoing work to

evaluate it\'s performance (and the performance of our interface in

general) for fine and course grained access to mesh (data) entities -

something that I suspect will be of interest to this group as well.

JohnC: I think it\'s premature to say. We need to have agreement on the

questions below first.

Ilmi: There will be some overhead and inefficiency using accessors for
data

exchange, but I like the apporach of accessors and believe the CCA
achieves

the reusability in expense of performance as OOP does anyway. Just we
try to

make the expense as little as possible.

JohnS: Given the focus on inter-component data exchange, I think
accessors provide the most straightforward paradigm for data exchange.
The arguments to the data access methods can involve elemental data
types rather than composite data structures (eg. we use scalars and
arrays of basic machine data types rather than hierarchical structures).
Therefore we should look closely at FM\'s API organization as well as
the accessors employed by SCIRun V1 (before they employed dynamic
compilation).

The accessor method works well for abstracting component location, but
requires potentially redundant copying of data for components in the
same memory space. It may be necessary to use reference counting in
order to reduce the need to recopy data arrays between co-located
components, but I\'d really like to avoid making ref counting a
mandatory requirement if we can avoid it. (does anyone know how to avoid
redundant data copying between opaque components without employing
reference counting?)

Wes: They are aligned to a large degree - data structures/models are
produced and

consumed by component code, but may also be manipulated (serialized,

marshalled, etc) by the framework.

What are requirements for the data representations that must be
supported by a common infrastructure. We will start by answering Pat\'s
questions of about representation requirements and follow up with
personal experiences involving particular domain scientist\'s
requirements.

> Must: support for structured data

Randy: Must-at the coarse level, I think this could form the basis of
all

other representations.

Pat: Structured data support is a must.

JohnC: Must

Jim: Must

JohnS: Must

Wes: Agree.

> Must/Want: support for multi-block data?

Randy: Must-at the coarse level, I think this is key for scalability,

domain decomposition and streaming/multpart data transfer.

Pat: We have unstructured data, mostly based on tetrahedral or prismatic
meshes.

We need support for at least those types. I do not think we could simply

graft unstructured data support on top of our structured data
structures.

JohnC: Must

Jim: Must

JohnS: Must

Wes: Must. We must set targets that meet our needs, and not sacrifice

requirements for speed of implementation.

> Must/Want: support for various unstructured data representations?
> (which ones?)

Randy: Nice-but I would be willing to live with an implementation on top

of structured, multi-block (e.g. Exdous). I feel accessors are

fine for this at the \"framework\" level (not at the leaves).

Pat: We have unstructured data, mostly based on tetrahedral or prismatic
meshes.

We need support for at least those types. I do not think we could simply

graft unstructured data support on top of our structured data
structures.

JohnC: Not sure. Not a priority.

Jim: Want (low priority)

JohnS: Cell based initially unstructured representations first. Need
support for arbitrary connectivity eventually, but not mandatory. I
liked Iris Explorer's hierarchical model as it seems more general than
the model offered by other vis systems.

Wes: Must. Unstructured data reps are widely used and they should not be

excluded from the base set of DS/DM technologies.

> Must/Want: support for adaptive grid standards? Please be specific
> about which adaptive grid methods you are referring to. Restricted
> block-structured AMR (aligned grids), general block-structured AMR
> (rotated grids), hierarchical unstructured AMR, or non-hierarchical
> adaptive structured/unstructured meshes.

Randy: Similar to my comments on unstructured data reps. In the long

run, something like boxlib with support for both P and H adaptivity

will be needed (IMHO, VTK might provide this).

Pat: Adaptive grid support is a \"want\" for us currently, probably
eventually

a \"must\". The local favorite is CART3D, which consists of hierarchical

regular grids. The messy part is that CART3D also supports having

more-or-less arbitrary shapes in the domain, e.g., an aircraft fuselage.

Handling the shape description and all the \"cut cell\" intersections

I expect will be a pain.

JohnC: Adaptive grid usage is in its infancy at NCAR. But I suspect it
is the

way of the future. Too soon to be specific about which adaptive grid

methods are prefered.

Jim: Want (low priority) the AMR folks havfe been trying to get together
and define

a standard API, and have been as yet unsuccessful. Who are we to attempt

this where they have failed\...?

JohnS: If we can define the data models rigorously for the individual
grid types (ie. structured and unstructured data), then adaptive grid
standards really revolve around an infrastructure for indexing data
items. We normally think of indexing datasets by time and by data
species. However, we need to have more general indexing methods that can
be used to support concepts of spatial and temporal relationships.
Support for pervasive indexing structures is also important for
supporting other visualization features like K-d trees, octrees, and
other such methods that are used to accelerate graphics algorithms. We
really should consider how to pass such representations down the data
analysis pipeline in a uniform manner because they are used so commonly.

Wes: Want, badly. We could start with Berger-Colella AMR since it is
widely

used. I\'m not crazy about Boxlib, though, and hope we can do something

that is easier to use.

> Must/Want: \"vertex-centered\" data, \"cell-centered\" data?
> other-centered?

Randy: Must.

Pat: Most of the data we see is still vertex-centered. FM supports other

associations, but we haven\'t used them much so far.

Jim: Want (low priority)

All of these should be \"Wants\", to the extent that they require more

sophisticated handling, or are less well-known in terms of generalizing

the interfaces.

For example, the AMR folks havfe been trying to get together and define

a standard API, and have been as yet unsuccessful. Who are we to attempt

this where they have failed\...?

So to clarify, if we \*really\* understand (or think we do) a particular

data representation/organization, or even a specific subset of a general

representation type, then by all means lets whittle an API into our
stuff.

Otherwise, leave it alone for someone else to do, or do as strictly
needed.

JohnS: The accessors must understand (or not preclude) all centering.
This is particularly for structured grids where vis systems are
typically lax in storing/representing this information.

Wes: Don\'t care - will let someone else answer this.

Note: It sounds like at least time-varying data handling is well
understood by the people who want it.

> Must: support time-varying data, sequenced, streamed data?

Randy: Must, but way too much to say here to do it justice. I will say

that the core must deal with time-varying/sequenced data. Streaming

might be able to be placed on top of that, if it is designed

properly. I will add that we have a need for progressive data as

well.

Pat: Support for time-varying data is a must.

JohnC: Must. Time varying data is what makes so many of our problems
currently

intractible. Too many of the available tools (e.g. VTK) assume static

data and completely fall apart when the data is otherwise.

Definitely there is

not support for any routines that require temporal integration (e.g.

unsteady flow viz). In general, there is no notion of a timestep in

VTk. Datasets are 3D. Period.

Additionally, there is a performance issue: VTK is not optimized in any

way at moving data through the pipeline at high rates. It\'s underlying

archictecture seems to assume that as long as the pipeline eventually

generates some geometry, it\'s ok if it take a loooong time because

you\'re going to interact with that geometry (navigating through camera

space, color space, etc.) and the \"pre-processing\" doesn\'t have to
run

at interactive rates. So the data readers are pathetically slow (and

there is little hope for optimization here with that data model that is

used). There is no way to exploit temporal coherence in any of the data

operators. No simple way to cache results if you want to play out of

memory.

At a high level you need a design that gives consideration to temporal
needs

throughout the architecture. I think the data structures do need to be

time varying data aware, not just capable of dealing with 4D data

(although I can\'t think of a specific example of why now). One issue is

that the temporal dimension often has different spacing/regularity than

the spatial dimension. Obviously you\'re talking different units from

the spatial dimensions as well. There are also system-level issues as

well (e.g. unsteady flow viz needs, exploiting temporal coherence,

caching, support for exploring the temporal dimension from user

interactors).

I know i\'ve only just started to scratch the surface here. We could

probably devote an entire workshop to time varying data needs and

several more figuring out how to actually suppor them.

Jim: MUST

JohnS: Yes to all. However, the concept of streamed data must be defined
in more detail. This is where the execution paradigm is going to affect
the data structures.

Wes: Not ready for prime time. I\'ve read two or three research
proposals in

the past year that focus on methods for time-varying data
representations

and manipulation. IMO, this topic is not ready for prime time yet. We

can say that it would be nice to have, but will probably not be fully

prepared to start whacking out code.

*Note: Should do quick gap-analysis on what existing tools fulfill this
requirement.*

> Must/Want: higher-order elements?

Randy: Must - but again, this can often be \"faked\" on top of other
reps.

Pat: Occasionally people ask about it, but we haven\'t found it to be a
\"must\".

JohnC: low priority

Jim: Wants, see above\...

JohnS: Not yet.

Wes: Not sure what this means, exactly, so I\'ll improvise. Beyond
scientific

data representations, there is a family of \"vis data structures\" that
need

to be on the table. These include renderable stuff - images, deep
images,

explicit and implicit geometry, scene graph ordering semantics, scene

specification semantics, etc. In addition, there is the issue of

\"performance data\" and how it will be represented.

*Note: I find the response to this quite funny because I ran a two day
workshop about 3 years ago here at LBNL on finite element analysis
requirements. We got bashed for two days straight by the FEM code jocks
because we didn't seem to care about higher order elements. So it would
be interesting to know if we don't see much of this because its not
needed or if the domain scientists simply lost all confidence in us to
deal with this issue properly.*

> Must/Want: Expression of material interface boundaries and other
> special-treatment of boundary conditions.

Randy: Must, but I will break this into two cases. Material interfaces
for

us are essentially sparse vectors sets, so they can be handled with

basic mechanisms so I do not see that as core, other than perhaps

support for compression. Boundary conditions (e.g. ghost zoning,

AMR boundaries, etc) are critical.

Pat: We don\'t see this so much. \"Want\", but not must.

JohnC: no priority

Jim: Want, see above...

JohnS: Yes, we must treat ghost zones specially or parallel vis
algorithms will create significant artifacts. I\'m not sure what is
required for combined air-ocean models.

Wes: I\'ll let someone else answer this one.

*Note: DOE vis workshop, it was pointed out that simple things like
isosurface give inconsistent (or radically different) results on
material interface boundaries depending on assumptions about the
boundary treatment. You'd think that this would come up with analysis
combined air-ocean models, but apparently not among the vis people. From
a data analysis standpoint, domain scientists say this is incredibly
important, but they can't deal with it because none of the vis or data
analysis people listen to them.*

> \* For commonly understood datatypes like structured and unstructured,
> please focus on any features that are commonly overlooked in typical
> implementations. For example, often data-centering is overlooked in
> structured data representations in vis systems and FEM researchers
> commonly criticize vis people for co-mingling geometry with topology
> for unstructured grid representations. Few datastructures provide
> proper treatment of boundary conditions or material interfaces. Please
> describe your personal experience on these matters.

Randy: Make sure you get the lowest common denominator correct! There is

no realistic way that the framework can support everything, everywhere

without losing its ability to be nimble (no matter what some OOP folks

say). Simplicity ore representation with \"externally\" supplied
optional

optimization information is one approach to this kind of problem.

Pat: One thing left out of the items above is support for some sort of
\"blanking\"

mechanism, i.e., a means to indicate that the data at some nodes are not

valid. That\'s a must for us. For instance, with Earth science data we
see

the use of some special value to indicate \"no data\" locations.

JohnC: Support for missing data is essential for observed fields.

To do it right you need some way to flag

data cells/vertices within the data model as not containing valid data.

Then you need to add support to your data \"operators\" as well. For
example,

if your operator is some kind of reconstruction filter it needs to know

to use a different kernel when missing data are involved.

Obviously, this could pose a signficant amount of overhead on the entire

system, and the effort may not be justified if the DOE doesn\'t have

great need for dealing with instrument acquired data. I only added the

point as a discussion topic as it is fairly important to us. At the

very least, I would hope to have the flexibility to hack support

for missing data if it was not integral to the core framework.

Jim: I don\'t think we should \"pee in this pool\" either yet. Are any
of us

experts in this kind of viz? Let\'s stick with what we collectively know

best and make that work before we try to tackle a
related-but-fundamentally-

different-domain.

JohnS: There is little support for non-cartesian coordinate systems in
typical data structures. We will need to have a discussion of how to
support coordinate projections/conversions in a comprehensive manner.
This will be very important for applications relating to the National
Virtual Observatory.

Wes: No comment

> \* Please describe data representation requirements for novel data
> representations such as bioinformatics and terrestrial sensor
> datasets. In particular, how should we handle more abstract data that
> is typically given the moniker \"information visualization\".

Randy: Obviously, do not forget \"records\" and aggregate/derived types.
That

having been said, the overheads for these can be ugly. Consider

parallel arrays as an alternative\...

Pat: \"Field Model\" draws the line only trying to represent fields and
the meshes

that the fields are based on. I not really familiar enough with other
types

of data to know what interfaces/data-structures would be best. We
haven\'t

see a lot of demand for those types of data as of yet. A low-priority
\"want\".

JohnC: Beats me.

JohnS: I simply don\'t know enough about this field to comment.

Wes: Maybe I don\'t understand the problem\...the same tough issues that
plague

more familiar data models appear to be present in bioinformatics and

\"info viz\" data mgt. There are heirarchical data, unstructured data,

multivariate and multidimensional data, etc.

*Note: Must separate mesh from field data interfaces.*

*The mesh may not be updated as often as the field.*

Perhaps time-range of validity is important information.

> What do you consider the most elegant/comprehensive implementation for
> data representations that you believe could form the basis for a
> comprehensive visualization framework?

-   For instance, AVS uses entirely different datastructures for
    structure, unstructured and geometry data. VTK uses class
    inheritance to express the similarities between related structures.
    Ensight treats unstructured data and geometry nearly interchangably.
    OpenDX uses more vector-bundle-like constructs to provide a more
    unified view of disparate data structures. FM uses data-accessors
    (essentially keeping the data structures opaque).

Randy: IMHO: layered data structuring combined with data accessors is

probably the right way to go. Keep the basic representational

elements simple.

Pat: Well, as you\'d expect, as the primary author of Field Model (FM) I
think it\'s

the most elegant/comprehensive of the lot. It handles structured and

unstructured data. It handles data non-vertex-centered data. I think it

should be able to handle adaptive data, though it hasn\'t actually been

put to the test yet. And of course every adaptive mesh scheme is a
little

different. I think it could handle boundary condition needs, though
that\'s

not something we see much of.

JohnC: I don\'t think this is what you\'re after, but i\'ve come to
believe that

multiresolution data representations with efficient domain subsetting

capabilities are the most pragmatic and elegant

way to deal with large data sets. In addition to enabling interaction

with the largest data sets they offer tremenous scalability from desktop

to \"visual supercomputer\". i would encourage a data model that
includes

and facilitates their integral support.

Ilmi: Combination of (externally) FM data-accessors and (internally) VTK
class

inheritance.

JohnS: Since I\'m already on record as saying that opaque data accessors
are essential for this project, it is clear that FM offers the most
compelling implementation that satisfies this requirement.

> \* Are there any of the requirements above that are not covered by the
> structure you propose?

Randy: I think one big issue will be the distributed representations.

This item is ill handled by many of these systems.

Pat: Out-of-core? Derived fields? Analytic meshes (e.g., regular
meshes)?

Differential operators? Interpolation methods?

JohnC: Not sure.

JohnS: We need to be able to express a wider variety of data layout
conversions and have some design pattern that reduces the need to recopy
data arrays for local components. The FM model also needs to have
additional API support for hierarchical indices to accelerate access to
subsections of arrays or domains.

Wes: Not sure how to answer. The one thing that came to mind is a
general observation

that the above data models are designed for scientific data. The AVS
geom

data structure was opaque to the developer, and if you looked at the
header

files, was really, really ugly. Since I have a keen interest in
renderers,

I am very concerned about having adequate flexibility and performance
from

a DS/DM for moving/representing renderable data, as opposed to large

structured or unstructured meshes. It is possible to generalize a

DS for storing renderable data (e.g, a scene graph), but this separate
class

of citizen reflects the partitioning of data types in AVS. Perhaps this

isn\'t something to be concerned about at this point.

*Note: Area of unique features?*

*-blanking arrays*

*-data handling for distributed data*

*-better handling of time-varying data*

*-hints for caching so that temporal locality can be exploited*

-indexing (not in TSTT sense... needs more discussion. Need support for
kD trees and rapid lookup. Indexing might help with our AMR issues)

> \* This should focus on the elegance/usefulness of the core
> design-pattern employed by the implementation rather than a
> point-by-point description of the implemenation!

Randy: Is it possible to consider a COM Automation Object-like approach,

also similar to the CCA breakdown. Basically, define the common

stuff and make it interchangable then build on top. Allow underlying

objects to be \"aware\" and wink to each other to bypass as needed.

In the long run, consider standardizing on working bypass paradigms

and bring them into the code (e.g. OpenGL).

Note: We need the "bypass". The question is how do we supply the bypass
mechanism for unanticipated data?

Pat: I think if we could reasonably cover the (preliminary) requirments
above,

that would be a good first step. I agree with Randy that whatever we

come up with will have to be able to \"adapt\" over time as our
understanding

moves forward.

> \* Is there information or characteristics of particular file format
> standards that must percolate up into the specific implementation of
> the in-memory data structures?

Randy: Not really, but metadata handling and referencing will be key and
need

to be general.

Pat: In FM we tried hard to file-format-specific stuff out of the core
model.

Instead, there are additional modules built on top of FM that handle

the file-format-specific stuff, like I/O and derived fields specific to

a particular format. Currently we have PLOT3D, FITS, and HDFEOS4

modules that are pretty well filled out, and other modules that are

mostly skeletons at this point.

We should also be careful not to assume that analyzing the data starts

with \"read the data from a file into memory, \...\". Don\'t forget
out-of-core,

analysis concurrent with simulation, among others.

One area where the file-format-specific issues creep in is with
metadata.

Most file formats have some sort of metadata storage support, some much

more elaborate than others. Applications need to get at this metadata,

possibly through the data model, possibly some other way. I don\'t have

the answer here, but it\'s something to keep in mind.

Jim: I dunno, but what does HDF5 or NetCDF include? We should definitely
be

able to handle various meta-data\...

Otherwise, our viz framework should be able to read in all sorts of

file-based data as input, converting it seamlessly into our \"Holy Data

Grail\" format for all the components to use and pass around. But the

data shouldn\'t be identifiable as having once been HDF or NetCDF,
etc\...

(i.e. it\'s important to read the data format, but not to use it
internally)

JohnS: I hope not.

Wes: One observation is what seems to be a successful design pattern
from the

DMF effort: let the HDF guys build the heavy lifting machinery, and
focus

upon an abstraction layer that uses the machinery to move bytes.

*Note: Metadata: Must also be propagated down the pipeline.*

*Ignored by items that don't care, but recognized by pipeline components
that do.*

*-alternative is database at the reader, but seems to create painful
connection mechanics.*

*-and still have to figure out how to reference the proper component,
even after going through data-structure transformations.*

*One powerful feature of both HDF and XML is the ability to ignore and
pass-through unrecognized constructs/metadata.*

> For the purpose of this survey, \"data analysis\" is defined broadly
> as all non-visual data processing done \*after\* the simulation code
> has finished and \*before\* \"visual analysis\".
>
> \* Is there a clear dividing line between \"data analysis\" and
> \"visual analysis\" requirements?

Randy: Not in my opinion.

Pat: Your definition excludes concurrent analysis and steering from

\"visualization\". Is this intentional? I don\'t think there\'s a clear
dividing

line here.

JohnC: I take issue with your definition of data analysis. Yes it is
performed

after the simulation, but it is performed (or would be performed if viz

tools didn\'t suck) in \*parallel\* with visual analysis. The two when

well integrated, which is rarely the case, can compliment each other

tremendously. So called \"visual analysis\" by itself, without good

quantitative capablity, is pretty useless.

Well, text based, programmable user interfaces are a must for \"data

analysis\" , whereas GUI is essential for visual.

Jim: NO. There shouldn\'t be - these operations are tightly coupled, or
even

symbiotic, and \*should\* all be incorporated into the same framework,

indistinguishable from each other.

Ilmi: Some components do purely data analysis, some do only visual, but
there

will be calls to the data analysis component from the visual during the

analysis.

JohnS: There shouldn\'t be. However, people at the SRM community left me
with the impression that they felt data analysis had been essentially
abandoned by the vis community in favor or \"visual analysis\" methods.
We need to undo this.

Wes: Generally speaking, there\'s not much difference.

That said, some differences seem obvious to me:

1\. Performance - visualization is most often an interactive process,
but

has offline implementations. \"Plain old\" data analysis seems to be
mostly

an offline activity with a few interactive implementations.

2\. Scope - data analysis seems to be a subset of vis. Data analysis
doesn\'t

have need for as rich a DS/DM infrastructure as vis.

Note: Righteous indignation. Well that's good. Why then do SDM people
and domain scientists think we don't care? They aren't smoking crack.
They have legitimate reasons to believe that we aren't being genuine
when we say that we care about data analysis functionality. Can I do
data analysis with Vis5D? Does VTK offer me a wide array of statistical
methods? Must keep this central as we design this system. Do you agree
with John Clyne's assertion that data analysis == text interface and
visualization==GUI.

> \* Can we (should we) incorporate data analysis functionality into
> this framework, or is it just focused on visual analysis.

Randy: Yes and we should, particularly as the complexity and size of

data grows, we begin to rely more heavily on \"data analysis\" based

visualization.

Pat: I think you would also want to include feature detection
techniques. For

large data analysis in particular, we don\'t want to assume that the
scientist

will want to do the analysis by visually scanning through all the data.

JohnC: If visualization is ever going to live up to the claim made by so
many

in the viz community of

it being an indispensable tool for analsyis, tight integration with

statistical tools and data processing capabilities are a must. Otherwise

we\'ll just continue to make pretty pictures, put on dog and pony shows,

and wonder where the users are.

Jim: YES.

Ilmi: Not all data analysis, but there are lots of data analysis being
used for

visual analysis and, the more tools are provided initially, it gets
easier

to make user-group become big. So, we can list candidates.

JohnS: Vis is bullshit without seamless integration with flexible data
analysis methods. The most flexible methods available are text-based.
The failure to integrate more powerful data analysis features into
contemporary 3D vis tools has been a serious problem.

Wes: Ideally, the same machinery could be used in both domains.

Notes: Data Analysis and Feature detection support constitutes more
unique features for this framework.

*How do you then index your bag of features or have them properly refer
back to the*

*Data that led to their generation? Sometimes detected feature is
discrete marker,*

*Other times, it is treated as a derived field. Former case seems to
again point to*

*Need for robust indexing method.*

*JohnC observation about data-analysis==text-based s very interesting!!!
Does everyone agree?*

*Aspect is one of few examples of providing a graphical workflow
interface for traditionally procedural/text-based data analysis tools.*

> \* What kinds of data analysis typically needs to be done in your
> field? Please give examples and how these functions are currently
> implemented.

Randy: Obviously basic statistics (e.g. moments, limits, etc).
Regression

and model driven analysis are common. For example, comparison of

data/fields via comparison vs common distance maps. Prediction of

activation \"outliers\" via general linear models applied on an

element by element basis, streaming through temporal data windows.

Pat: Around here there is interest in vector-field topology feature
detection

techniques, for instance, vortex-core detection.

JohnC: Pretty much everything you can do with IDL or matlab.

Jim: Simple sampling, basic statistical averages/deviations, principal
component

analysis (PCA, or EOF for climate folks), other dimension reduction.

Typically implemented as C/C++ code\... mostly slow serial\... :-Q

JohnS: This question is targeted at vis folks that have been focused on
a particular scientific domain. For general use, I think of IDL as being
one of the most popular/powerful data analysis languages. Python has
become increasingly important \-- especially with the Livermore
numerical extensions and the PyGlobus software. However, use of these
scripting/data analysis languages have not made the transition to
parallel/distributed-memory environments (except in a sort of
data-parallel batch mode).

> \* How do we incorporate powerful data analysis functionality into the
> framework?

Randy: Hard work :), include support for meta-data, consider support for

sparse data representations and include the necessary support for

\"windowing\" concepts.

Pat: Carefully :-)? By striving not to make a closed system.

JohnC: I\'d suggest exploring leveraging existing tools, numerical
python for

example.

Jim: As components (duh)\... :-)

We should define some \"standard\" APIs for the desired analysis
functions,

and then either wrap existing codes as components or shoehorn in
existing

component implementations from systems like ASPECT.

JohnS: I\'m very interested in work that Nagiza has proposed for a
parallel implementation of the R statistics language. The traditional
approach for parallelizing scripting languages is to run them in a sort
of MIMD mode of Nprocs identical scripts operating on different chunks
of the same dataset. This makes it difficult to have a
commandline/interactive scripting environment. I think Nagiza is
proposing to have an interactive commandline environment that
transparently manipulates distributed actions on the back-end.

There is a similar work in progress on parallel matlab at UC Berkeley.
Does anyone know of such an effort for Python? (most of the parallel
python hacks I know of are essentially MIMD which is not very useful).

> 2\) Execution Model=======================

It will be necessary for us to agree on a common execution semantics for
our components. Otherwise, while we might have compatible data
structures but incompatible execution requirements. Execution semantics
is akin to the function of protocol in the context of network
serialization of data structures. The motivating questions are as
follows;

-   How is the execution model affected by the kinds of
    algorithms/system-behaviors we want to implement.

Pat: In general I see choices where at one end of the spectrum we have

simple analysis techniques where most of the control responsibilities

are handled from the outside. At the other end we could have more

elaborate techniques that may handle load balancing, memory

management, thread management, and so on. Techniques towards

the latter end of the spectrum will inevitably be intertwined more

with the execution model.

Jim: Directly. There are probably a few main exec models we want to
cover.

I don\'t think the list is \*that\* long\...

As such, we should anticipate building several distinct framework

environments that each exclusively support a given exec model. Then

the trick is to \"glue\" these individual frameworks together so they
can

interoperate (exchange data and invoke each others\' component methods)

and be arbitrarily \"bridged\" together to form complex higher-level

pipelines or other local/remote topologies.

Ilmi: I guess we can make each component propagate/fire the execution of
next

component/components in the network/pipeline. Each component can use
their

own memory or shared memory to access the data in process. In such case,

algorithm of each component does not get much affected by other
coponents

around.

Wes: The \"simple\" execution model is for the framework to invoke a
component, be

notified of its completion, then invoke the next component in the chain,
etc.

Things get more interesting if you want to have a streaming processing
model.

Related, progressive processing is somewhat akin to streaming, but more

stateful.

Note: Wes' model sounds like a good "baseline" model. It does not allow
for chains-of-invokation and therefore prevents us from getting
locked-in to complex issues of comonent-local execution semantics and
deadlock prevention. Can we make this a baseline component requirement?

-   How then will a given execution model affect data structure
    implementations

Pat: Well, there\'s always thread-safety issues.

Jim: I don\'t think it should affect the data structure impls at all,
per se.

Clearly, the access patterns will be different for various execution
models,

but this shouldn\'t change the data impl. Perhaps a better question is

how to indicate the expected access pattern to allow a given data impl

to optimize or properly prefetch/cache the accesses\...

*Note: Actually, that is the question. How do we pass information about
access patterns so that you can do the kind of temporal caching that
John Clyne wants? Its important to **not** do what VTK does, so that we
don't have to un-do it again (as was the case for VisIt).*

Ilmi: I guess we can make each component propagate/fire the execution of
next

component/components in the network/pipeline. Each component can use
their

own memory or shared memory to access the data in process. In such case,

algorithm of each component does not get much affected by other
coponents

around.

JohnS: There will need to be some way to support both declarative
execution semantics, data-driven and demand-driven semantics. By
declarative semantics, I mean support for environments that want to be
in control of when the component \"executes\" or interactive scripting
environments that wish to use the components much like subroutines. This
is separate from the demands of very interactive use-cases like
view-dependent algorithms where the execution semantics must be more
automatic (or at least hidden from the developer who is composing the
components into an application). I think this is potentially relevant to
data model discussions because the automatic execution semantics often
impose some additional requirements on the data structures to hand off
tokens to one another. There are also issues involved with managing
concurrent access to data involved. For instance, a demand-driven system
demanded of progressive-update or view-dependent algorithms, will need
to manage the interaction between the arrival of new data and
asynchronous requests from the viewer to recompute existing data as the
geometry is rotated.

(note: Wes provides a more succinct description of this execution
semantics.)

Wes: We\'re back to the issue of needing a DS/DM that supports
multiresolution

models from the git-go. The relationship between data analysis and vis
data

models becomes more apparent here when we start thinking about multires

representations of unstructured data, like particle fields or point
clouds.

*Note: The fly in the ointment here is that highly interactive methods
like multires models and view-dep algorithms are not well supported by
completely simple/declarative semantics (unless you have an incredibly
complex framework, but then the framework would require
component-specific knowledge to schedule things properly).*

-   How will the execution model be translated into execution semantics
    on the component level. For example will we need to implement
    special control-ports on our components to implement particular
    execution models or will the semantics be implicit in the way we
    structure the method calls between components.

Pat: Not sure.

Jim: Components should be \"dumb\" and let other components or the
framework invoke

them as needed for a given execution model. The framework dictates the

control flow, not the component. The API shouldn\'t change.

If you want multi-threaded components, then the framework better support

that, and the API for the component should take the possibility into
account.

JohnS: I\'m going to propose that we go after the declarative semantics
first (no automatic execution of components) with hopes that you can
wrap components that declare such an execution model with your own
automatic execution semantics (whether it be a central executive or a
distributed one). This follows the paradigm that was employed for tools
such as VisIt that wrapped each of the pieces of the VTK execution
pipeline so that it could impose its own execution semantics on the
pipeline rather than depending on the exec semantics that were
predefined by VTK. DiVA should follow this model, but start with the
simplest possible execution model so that it doesn\'t need to be
deconstructed if it fails to meet the application developer\'s needs (as
was the case with VisIt).

We should have at least some discussion to ensure that the \*baseline\*
declarative execution semantics imposes the fewest requirements for
component development but can be wrapped in a very
consistent/uniform/simple manner to support any of our planned pipeline
execution scenarios. This is an excercise in making things as simple as
possible, but thinking ahead far enough about long-term goals to ensure
that the baseline is \"future proof\" to some degree.

Wes: One thing that always SUPREMELY annoyed me about AVS was the
absence of a

\"stop\" button on the modules. This issue concerns being able to
interrupt

a module\'s processing when it was taking too long. Related, it might be
nice

to have an execution model that uses the following paradigm: \"OK,
time\'s up,

give me what you have now.\"

*Note: That is another way that the exec model is going to affect
structures (or at least the API for accessing those structures). We can
refer to this as "concurrent access to data". Do we have to incorporate
locking semantics in the data accessors? Do we have to incorporate
firing protocol into the accessors (or at least hints as to firing
constraints)? Again, we don't want to get into a VisIt situation.*

*Automatic exec semantics do not give the framework enough control to
address Wes' issue. Certainly this is an issue with VTK as well. How do
we formulate this as a requirement?*

> What kinds of execution models should be supported by the distributed
> visualization architecture
>
> \* View dependent algorithms? (These were typically quite difficult to
> implement for dataflow visualization environments like AVS5).

Randy: I propose limited enforcement of fixed execution semantics.
View/data/focus

dependent environments are common and need to be supported, however,
they

are still tied very closely with data representations, hence will likely

need to be customized to application domains/functions.

Pat: Not used heavily here, but would be interesting. A \"want\".

JohnC: These are neat research topics, but i\'ve never been convinced
that they

have much application beyond IEEEViz publications. Mostly I believe

this because of the complexity they impose on the data model. Better to

simply offer progressive/multiresolution data access.

Jim: Want.

Ilmi: I like to say \"must\", but it is for improving usability and
efficiency,

so people may live without it.

It will definitely improve the efficiency. If we want to support view

dependent algorithm, then we should consider it from the beginning of
the

dataflow design, so it can be easily integrated into. View dependent or

image-based algorithm doesn\'t necessarily make much changes to existing
data

flow design. View dependant or image-based algorithms are useful to

eliminate majority of data blocks from the rendering pipeline.
Therefore, it

is good to provide capability to choose subset of data to be rendered
from

the dataflow.

JohnS: Must be supported, but not as a basline exec model.

Wes: Yes

# Note: Is this a framework or component issue? Or is it a job for hierarchical components?

> \* Out-of-core algorithms

Randy: This has to be a feature, given the focus on large data.

Pat: A \"must\" for us.

JohnC: Seems like a must for large data. But is this a requirement or a
design

issue?

Jim: Must. This is a necessary evil of \"big data\". You need some
killer

caching infrastructure throughout the pipeline (e.g. like VizCache).

JohnS: Same deal. We must work out what kinds of attributes are required
of the data structures/data model to represent temporal decomposition of
a dataset. We should not encode the execution semantics as part of this
(it should be outside of the component), but we must ensure that the
data interfaces between components are capable of representing this kind
of data decomposition/use-case.

Wes: Yes

> \* Progressive update and hierarchical/multiresolution algorithms?

Randy: Obviously, I have a bias here, particularly in the remote
visualization

cases. Remote implies fluctuations in effective data latency that make

progressive systems key.

Pat: A \"want\".

JohnC: This is the way to go (IMHO), the question is at what level to
support

it.

Jim: Must

Ilmi: MUST! for improving usability and efficiency. And can be used to
support

view-dependent algorithm.

JohnS: Likewise, we should separate the execution semantics necessary to
implement this from the requirements imposed on the data representation.
Data models in existing production data analysis/visualization systems
often do not provide an explicit representation for such things as
multiresolution hierarchies. We have LevelOfDetail switches, but that
seems to be only a week form of representation for these hierarchical
relationships and limits the effectivness of algorithms that depend on
this method of data representation. Those requirements should not be
co-mingled with the actual execution semantics for such components (its
just the execution interface)

Wes: Yes. All of the above. Go team!

> \* Procedural execution from a single thread of control (ie. using an
> commandline language like IDL to interactively control an dynamic or
> large parallel back-end)

Randy: Yep, I think this kind of control is key.

Pat: A \"want\".

JohnC: A must for data analysis and data manipulation (derving new
fields, etc)

Jim: This is not an execution model, it is a command/control interface
issue.

You should be able to have a GUI, programmatic control, or scripting to

dictate interactive control (or \"steering\" as they call it\... :-).
The

internal software organization shouldn\'t change, just the interface to

the outside (or inside) world\...

Ilmi: Good to have

JohnS: This should be our primary initial target. I do not have a good
understanding of how best to support this, but its clear that we must
ensure that a commandline/interactive scripting language must be
supported. Current data parallel scripting interfaces assume
data-parallel, batch-mode execution of the scripting interpreters (this
is a bad thing).

Wes: HIstorically, this approach has proven to be very useful.

> \* Dataflow execution models? What is the firing method that should be
> employed for a dataflow pipeline? Do you need a central executive like
> AVS/OpenDX or, completely distributed firing mechanism like that of
> VTK, or some sort of abstraction that allows the modules to be used
> with either executive paradigm?

Randy: I think this should be an option as it can ease some connection

mechanisms, but it should not be the sole mechanism. Personally,

I find a properly designed central executive making \"global\"

decisions coupled with demand/pull driven local \"pipelets\" that

allow high levels of abstraction more useful (see the VisIt model).

Pat: Preferably a design that does not lock us in to one execution
model.

JohnC: We use a wavelet based approach similar to space filling curves.
Both

approaches have merrit and both should be supportable by the framework.

Jim: Must This should be an implementation issue in the \"dataflow
framework\", and

should not affect the component-level APIs.

JohnS: This can probably be achieved by wrapping components that have
explicit/declarative execution semantics in a
"component-within-a-component" hierarchical manner. Its an open question
as to whether these execution models are a function of the component or
the framework that is used to compose the components into an application
though.

Wes: I get stuck thinking about the UI for this kind of thing rather
than

the actual implementation. I\'ll defer to others for opinions.

Note: Strange... I would have tagged SFC's and Wavelets as "researchy"
things.

> \* Support for novel data layouts like space-filling curves?

Randy: With the right accessors nothing special needs to be added for
these.

Pat: Not a pressing need here, as of yet.

Jim: Must. But this isn\'t an execution model either. It\'s a data
structure

or algorithmic detail\...

JohnS: I don\'t understand enough about such techniques to know how to
approach this. However, it does point out that it is essential that we
hand off data structures via accessors that keep the internal data
structures opaque rather than complex data structures.

-   Are there special considerations for collaborative applications?

Jim: Surely. The interoperability of distinct framework implementations

ties in with this\... but the components shouldn\'t be aware that they

are being run collaboratively/remotely\... definitely a framework issue.

Ilmi: Some locking mechanizm for subset of data or dispatching of
changes from

one client to multiple clients.

JohnS: Ugh. I\'m also hoping that collaborative applications only impose
requirements for wrapping baseline components rather than imposing
internal requirements on the interfaces that exchange data between the
components. So I hope we can have \"accessors\" or
\"multiplexor/demultiplexor\" objects that connect to essentially
non-collaboration-aware components in order support such things.
Otherwise, I\'m a bit daunted by the requirements imposed.

Note: The danger of pushing the collaborative functionality out to a
"framework issue" is that we increasingly make the "framework" a
heavyweight object. It creates a high-cost-of-entry for any such feature
or even minor modifications to such features. Learned from "Cactus" the
importance of making the framework as slender as possible and move as
much functionality as possible into "optional components" to support
feature X. So it is important to ensure that we push off these issues as
much as possible.

> \* What else?

Randy: The kitchen sink? :)

Pat: Distributed control? Fault tolerance?

Jim: Yeah Right.

Wes: Control data, performance data and framework response to and
manipulation

of such data.

> How will the execution model affect our implementation of data
> structures?

Jim: It shouldn\'t. The execution model should be kept independent of
the

data structures as much as possible.

If you want to build higher-level APIs for specific data access patterns

> that\'s fine, but keep the underlying data consistent where possible.

Note: The description of this as affecting our "data structures" is an
artifact of attempting to straddle the dual-goals of addressing internal
data structures and external accessors. So perhaps this should be "how
will it affect our accessors."

\* how do you decompose a data structure such that it is amenable to
streaming in small chunks?

Randy: This is a major issue and relates to things like out-of-core/etc.

I definitely feel that \"chunking\" like mechanisms need to be in

the core interfaces.

Pat: Are we assuming streaming is a requirement?

How do you handle visualization algorithms where the access patterns

are not known a priori? The predominant example: streamlines and
streaklines.

Note the access patterns can be in both space and time. How do you avoid

having each analysis technique need to know about each possible data

structure in order to negotiate a streaming protocol? How do add another

data structure in the future without having to go through all the
analysis

techniques and put another case in their streaming negotiation code?

In FM the fine-grained data access (\"accessors\") is via a standard

interface. The evaluation is all lazy. This design means more

function calls, but it frees the analysis techniques from having to know

access patterns a priori and negotiate with the data objects. In FM

the data access methods are virtual functions. We find the overhead

not to be a problem, even with relatively large data. In fact, the
overhead

is less an issue with large data because the data are less likely to be

served up from a big array buffer in memory (think out-of-core, remote

out-of-core, time series, analytic meshes, derived fields, differential-

operator fields, transformed objects, etc., etc.).

The same access-through-an-interface approach could be done without

virtual functions, in order to squeeze out a little more performance,
though

I\'m not convinced it would be worth it. To start with you\'d probably
end up

doing a lot more C++ templating. Eliminating the virtual functions would

make it harder to compose things at run-time, though you might be able

to employ run-time compilation techniques a la SCIRun 2.

Jim: This sounds a lot like distributed data decompositions. I suspect
that

given a desired block/cycle size, you can organize/decompose data in all

sorts of useful ways, depending on the expected access pattern.

In conjunction with this, you could also reorganize static datasets

into filesystem databases, with appropriate naming conventions or

perhaps a special protocol for lining up the data blob files in the

desired order for streaming (in either time or space along any axis).

Meta-data in the files might be handy here, too, if it\'s indexed

efficiently for fast lookup/searching/selection.

JohnS: The recent SDM workshop pointed out that chunking/streaming
interfaces are going to be essential for any data analysis system that
deals with large data, but there was very little agreement on how the
chunking should be expressed. The chunking also potentially involves
end-to-end requirements of the components that are assembled in a
pipeline as you must somehow support uniformity in the passage of chunks
through the system (ie. the decision you make about the size of one
chunk will impose requirements for all other dependent streaming
interfaces in the system). We will need to walk through at least one
use-case for chunking/streaming to get an idea of what the constraints
are here. It may be too tough an issue to tackle in this first meeting
though.

Also, as Pat pointed out, when dealing with vis techniques like
streamlines, you almost need to have a demand-based fetching of data.
This implies some automatic propagation of requests through the
pipeline. This will be hard, and perhaps not supported by a baseline
procedural model for execution.

Note: Again, it appears we need to have clear delineation between
temporal and spatial dependencies. To support streaming, one must also
have dependent components be able to report back their constraints.

*Jim, how can we formulate a requirement that the execution model is
independent of the data structures when we really don't have data
structures per-se. Because we are using accessors, calling them will in
turn cause a component to call other accessors. If we do not have common
execution semantics, then this will be a complete muddle even if we do
agree on our port standards. So can we really keep these things
independent?*

> \* how do you represent temporal dependencies in that model?

Randy: I need to give this more thought, there are a lot of options.

Pat: In FM, data access arguments have a time value, the field interface
is

the same for both static and time-varying data.

Jim: Meta-data, or file naming conventions\...

JohnS: Each item in a datastructure or as passed-through via an accessor
needs to have some method of referring to dependencies both spatial (ie.
interior boundaries caused by domain decomposition) and temporal. Its
important to make these dependencies explicit in the data structures
provide a framework the necessary information to organize parallelism in
both the pipeline and data-parallel directions. The implementation
details of how to do so are not well formulated and perhaps out-of-scope
for our discussions. So this is a desired \*requirement\* that doesn\'t
have a concrete implementation or design pattern involved.

*Note: Given the importance of time-varying data to JohnC and Pat, it
seems important to come up with a formal way to represent these things.*

> \* how do you minimize recomputation in order to regenerate data for
> view-dependent algorithms.

Randy: Framework invisible caching. Not a major Framework issue.

Pat: Caching? I don\'t have a lot of experience with view-dependent
algorithms.

Jim: No clue.

JohnS: I don\'t know. I\'m hoping someone else responding to this survey
has some ideas on this. I\'m uncertain how it will affect our data model
requirements.

*Note: Is caching a framework issue? Or is it a component issue?*

> What are the execution semantics necessary to implement these
> execution models?
>
> \* how does a component know when to compute new data? (what is the
> firing rule)

Randy: Explicit function calls with potential async operation. A
higher-level

wrapper can make this look like \"dataflow\".

Jim: There are really only 2 possibilities I can see - either a
component is

directly invoked by another component or the framework, or else a method

must be triggered by some sort of dataflow dependency or stream-based

event mechanism.

JohnS: For declarative semantics, the firing rule is an explicit method
call that is invoked externally. Hopefully such objects can be
\*wrapped\* to encode semantics that are more automatic (ie. the module
itself decides when to fire depending on input conditions), but
initially it should be explicit.

Wes: To review, the old AVS model said that a module would be executed
if any

of its parameters changed, or if its input data changed. One thing that

was annoying was that you had to explicitly disable the flow executive
if

you wanted to make changes to multiple parameters on a single module
before

allowing it to execute. This type of thing came up when using a module
with

a long execution time.

\* does coordination of the component execution require a central
executive or can it be implemented using only rules that are local to a
particular component.

Randy: I think the central executive can be an optional component
(again, see

VisIt).

Jim: This is a framework implementation detail. No. No. Bad Dog.

The component doesn\'t know what\'s outside of it (in the rest of the

framework, or the outside world). It only gets invoked, one way or

another.

JohnS: It can eventually be implemented using local semantics, but
intiially, we should design for explicit external control.

Wes: Not sure what this means.

Note: And it potentially invokes other components. If a component
invokes other components and thereby creates a chain of execution, then
we have an execution semantics that is outside of the framework's
control. So, do we want to prevent this in our baseline requirements for
component invocation? The central executive approach says that our
"baseline" components may not invoke another component in response to
their invocation. This seems to be a component invocation semantics
issue.

> \* how elegantly can execution models be supported by the proposed
> execution semantics? Are there some things, like loops or
> back-propagation of information that are difficult to implement using
> a particular execution semantics?

Randy: There will always be warts\...

Pat: The execution models we have used have kept the control model in

each analysis technique pretty simple, relying on an external executive.

The one big exception is with multi-threading. We\'ve experimented with

more elaborate parallelism and load-balancing techniques, motivated in

part by latency hiding desires.

Jim: We need to keep the different execution models separate, as
implementation

details of individual frameworks. This separates the concerns here.

JohnS: Its all futureware at this point. We want to first come up with
clear rules for baseline component execution and then can come up with
some higher level / automatic execution semantics that can be
implemented by \*wrapping\* such components. The \"wrapper\" would then
take responsibility for imposing higher-level automatic semantics.

Wes: The dataflow thing doesn\'t lend itself well to things like view
dependent

processing where the module at the end of the chain (renderer) sends
view

parameters back upstream, thereby causing the network to execute again,
etc.

The whole upstream data thing is a \"wart on the ass of\" AVS. (sorry)

> How will security considerations affect the execution model?

Randy: Security issues tend to impact two areas: 1) effective
bandwidth/latency

and 2) dynamic connection problems. 1) can be unavoidable, but will not

show up in most environments if we design properly. 2) is a real problem

with few silver bullets.

Pat: More libraries to link to? More latency in network communication?

Jim: Ha ha ha ha\...

They won\'t right away, except in collaboration scenarios.

Think \"One MPI Per Framework\" and do things the old fashioned way

locally, then do the \"glue\" for inter-framework connectivity with

proper authentication only as needed. (No worse than Globus\... :-)

JohnS: I don\'t know. Please somebody tell me if this is going to be an
issue. I don\'t have a handle on the \*requirements\* for security. But
I do know that simply using a secure method to \*launch\* a component is
considered insufficient by security people who would also require that
connections between components be explicitly authenticated as well. Most
vis systems assume secure launching (via SSH or GRAM) is sufficient. The
question is perhaps whether security and authorization are a framework
issue or a component issue. I am hoping that it is the former (the role
of the framework that is used to compose the components).

Note: Current DOE security policy basically dictates that we cannot
deploy current distributed vis tool implementations because the
connections are not authenticated. Ensight is an exception because the
server is always making an outgoing connection (basically makes it an
issue for the destination site) and requires explicit "accept" of the
connection.

> 3\) Parallelism and load-balancing=================
>
> Thus far, managing parallelism in visualization systems has been a
> tedious and difficult at best. Part of this is a lack of powerful
> abstractions for managing data-parallelism, load-balancing and
> component control.

JohnS: If we are going to address inter-component data transfers to the
exclusion of data structures/models internal to the component, then much
of this section is moot. The only question is how to properly represent
data-parallel-to-data-parallel transfers and also the semantics for
expressing temporal/pipeline parallelism and streaming semantics.
Load-balancing becomes an issue that is out-of-scope because it is
effectively something that is inside of components (and we don\'t want
to look inside of the components

> Please describe the kinds of parallel execution models that must be
> supported by a visualization component architecture.

-   data-parallel/dataflow pipelines?

# Jim: Must

JohnS: Must

Wes: It would be nice if the whole scatter/gather thing could be
marshaled

by the framework. That way, my SuperSlick\[tm\] renderer wouldn\'t
contain

a bunch of icky network code that manages multiple socket connections

from an N-way parallel vis component. One interesting problem is how a

persistent tool, like a renderer, will be notified of changes in data

originating from external components. I want some infrastructure that

will make obsolete me having to write custom code like this for each

new project.

Note: Seriously. Is it really a very useful paradigm to have the
framework represent parallel components as one-component-per-processor?
Its seems very 'icky' as Wes says.

> \* master/slave work-queues?

Randy: I tend to use small dataflow pipelines locally and higher-level

async streaming work-queue models globally.

Jim: Must

JohnS: Maybe: If we want to support progressive update or heterogeneous
execution environments. However, I usually don't consider this
methodology scalable.

> \* streaming update for management of pipeline parallelism?

Randy: Yes, we use this, but it often requires a global parallel
filesystem to

be most effective.

Jim: Must

JohnS: Must

> \* chunking mechanisms where the number of chunks may be different
> from the number of CPU\'s employed to process those chunks?

Randy: We use spacefilling curves to reduce the overall expense of this

(common) operation (consider the compute/viz impedance mismatch

problem as well). As a side effect, the codes gain cache coherency

as well.

Pat: We\'re pretty open here. Mostly straight-forward work-queues.

Jim: This sounds the same as master/slave to me, as in \"bag of
tasks\"\...

JohnS: Absolutely. Of course, this would possibly be implemented as a
master/slave work-queue, but there are other methods.

> \* how should one manage parallelism for interactive scripting
> languages that have a single thread of control? (eg. I\'m using a
> commandline language like IDL that interactively drives an arbitrarily
> large set of parallel resources. How can I make the parallel back-end
> available to a single-threaded interactive thread of control?)

Randy: Consider them as \"scripting languages\", and have most
operations

run through an executive (note the executive would not be aware

of all component operations/interactions, it is a higher-level

executive). Leave RPC style hooks for specific references.

Pat: I\'ve used Python to control multiple execution threads. The (C++)

data objects are thread safe, the minimal provisions for thread-safe

objects in Python haven\'t been too much of a problem.

Jim: Broadcast, Baby\... Either you blast the commands out to everyone
SIMD

style (unlikely) or else you talk to the Rank 0 task and the command

gets forwarded on a fast internal network.

JohnS: I think the is very important and a growing field of inquiry for
data analysis environments. Whatever agreements we come up with, I want
to make sure that things like parallel R are not left out in these
considerations.

*Note: But CCA doesn't support broadcast. This leads to a quandary here
because we want to be able to be able to adjust parameters for a
component via the GUI **or** via a command from another component
interchangeably. So I agree with "broadcast baby", but I don't see that
it is feasible to push this off as a "framework issue" as it may well
need to be something the component interface description must support.*

> Please describe your vision of what kinds of software support /
> programming design patterns are needed to better support parallelism
> and load balancing.
>
> \* What programming model should be employed to express parallelism.
>
> (UPC, MPI, SMP/OpenMP, custom sockets?)

Randy: The programming model must transcend specific parallel APIs.

Jim: All but UPC will be necessary for various functionality.

JohnS: If we are working just on the outside of components, this
question should be moot. We must make sure the API is not affected by
these choices though.

Wes: This discussion may follow the same path as the one about DS/DM for
grids.

The answer seems to be \"one size doesn\'t fit all, but there is no
\'superset\'

the makes everyone happy.\" That said, there is likely a set of common
issues

wrt execution and DS/DM that underly parallel components regardless of

implementation.

Note: Since we are talking about functionality outside of the component,
this seems reasonable. So this really requires clarification of where
the parallelism is expressed. Caffeine wants to express this as parallel
sets of components. However, this seems unreasonable for some of the
communication patterns we deal with. If we stated that such parallelism
is inside of the component wrapper, then what? (at minimum, we don't
have to answer this question!)

> \* Can you give some examples of frameworks or design patterns that
> you consider very promising for support of parallelism and load
> balancing.
>
> (ie. PNNL Global Arrays or Sandia\'s Zoltan)
>
> [http://www.cs.sandia.gov/Zoltan/]{.underline}
>
> [http://www.emsl.pnl.gov/docs/global/ga.html]{.underline}

Randy: no I cannot (am not up to speed).

Jim: Nope, that covers my list of hopefuls.

JohnS: Also out of scope. This would be something employed within a
component, but if we are restricting discussions to what happens on the
interface between components, then this is also a moot point. At
minimum, it will be important to ensure that such options will not be
precluded by our component interfaces.

Wes: Mabye should include \"remote resource management\" in this thread.
I\'m

thinking of the remote AVS module libraries. So, not only is there the
issue

of launching parallel components, and load balancing (not sure how this
will

play out), but also one of allowing a user to select, at run time, from

among a set of resources.

This problem becomes even more interesting when the pipeline
optimization

starts to happen, and components are migrated across resources.

Note: This is now somewhat out-of-scope for discussions of
inter-component communication.

\* Should we use novel software abstractions for expressing parallelism
or should the implementation of parallelism simply be an opaque property
of the component? (ie. should there be an abstract messaging layer or
not)

Randy: I would vote no as it will allow known paradigms to work, but
will

interfere with research and new direction integration. I think some

kind of basic message abstraction (outside of the parallel data system)

is needed.

Jim: It\'s not our job to develop \"novel\" parallelism abstractions. We
should

just use existing abstractions like what the CCA is developing.

JohnS: Implementation of parallelism should be an opaque property of the
component. We want to have language independence. We should also strive
to support independence in the implementation of parallelism. Creating a
software abstraction layer for messaging and shmem is a horrible way to
do it.

> \* How does the NxM work fit in to all of this? Is it sufficiently
> differentiated from Zoltan\'s capabilities?

Randy: Unable to comment\...

Pat: I don\'t have a strong opinion here. I\'m not familiar with Zoltan
et al.

Our experience with parallelism tends to be more shared-memory than

distributed memory.

JohnC: Hmm. These all seem to be implementation issues. Too early to
answer.

JohnS: I need a more concrete understanding of MxN. I understand what it
is supposed to do, but I\'m not entirely sure what requirements it would
impose on any given component interface implementation. It seems like
something our component data interfaces should support, but perhaps such
redistribution could be hidden inside of an MxN component? So should
this kind of redistribution be supported by the inter-component
interface or should there be components that explicitly effect such data
redistributions? Jim\... Help!

Jim: I don\'t know what Zoltan can do specifically, but MxN is designed
for

basic \"parallel data redistribution\". This means it is good for doing

big parallel-to-parallel data movement/transformations among two
disparate

parallel frameworks, or between two parallel components in the same

framework with different data decompositions. MxN is also good for

\"self-transpose\" or other types of local data reorganization within a

given (parallel) component.

MxN doesn\'t do interpolation in space or time (yet, probably for a
while),

and it won\'t wash your car (but it won\'t drink your beer either\...
:-).

If you need something fancier, or if you don\'t really need any data

reorganization between the source and destination of a transfer, then

MxN \*isn\'t\* for you\...

> ===============End of Mandatory Section (the rest is
> voluntary)=============
>
> 4\) Graphics and Rendering=================
>
> What do you use for converting geometry and data into images (the
> rendering-engine). Please comment on any/all of the following.
>
> \* Should we build modules around declarative/streaming methods for
> rendering geometry like OpenGL, Chromium and DirectX or should we move
> to higher-level representations for graphics offered by scene graphs?

Randy: IMHO, the key is defining the boundary and interoperability
constraints.

If these can be documented, then the question becomes moot, you can

use whatever works best for the job.

Ilmi: It is usually useful to have access to frame buffer so, I prefer
OpenGL

style over VRML style.

In addition, I don\'t know how useful the scene graphs for
visualization. I

guess scene graphs for visualizations are relatively simple, so it is

possible to convert the scene graphs to declarative way. So, mainly
support

declarative methods and then additional support of scen graphs and

conversions to declarative methods.

JohnS: This all depends on the scope of the framework. A-priori, you can
consider the rendering method separable and render this question moot.
However, this will make it quite difficult to provide very sophisticated
support for progressive update, image-based-methods, and view-dependent
algorithms because the rendering engine becomes intimately involved in
such methods. I\'m concerned that this is where the component model
might break down a bit. Certainly the rendering component of traditional
component-like systems like AVS or NAG Explorer the most heavy-weight
and complex components of the entire environment. Often, the
implementation of the rendering component would impose certain
requirements on components that had to interact with it closely
(particularly in the case of NAG/Iris Explorer where you were really
directly exposed to the fact that the renderer was built atop of
OpenInventor).

So, we probably cannot take on the issue of renderers quite yet, but we
are eventually going to need to define a big \"component box\" around
OpenGL/Chromium/DirectX. That box is going to have to be carefully built
so as to keep from precluding any important functionality that each of
those rendering engines can offer. Again, I wonder if we would need to
consider scene graphs if only to offer a persistent datastructure to
hand-off to such an opaque rendering engine. This isn\'t necessarily a
good thing.

Wes: As a scene graph proponent, I would say that you don\'t build
component

architectures around scene graphs. That concept doesn\'t make any sense
to me.

Instead, what you do is have DS/DM representations/encapsulations for
the

results of visualization. These are things like buckets-o-triangles,
perhaps

at multiple resolutions. You also provide the means to send renderer
information

to vis components to do view-dependent processing, or some other form of

selective processing.

Similarly, you don\'t make the output of visualization components in the
form

of glBegin()/glEnd() pairs, either.

*Note: It sounds like we need to look at Ilmi's work and ensure that
whatever method we select to get the "drawables" to the "renderer" that
it not preclude her requirements.*

> What are the pitfalls of building our component architecture around
> scene graphs?

Randy: Data cloning, data locking and good support for streaming, view
dependent,

progressive systems.

JohnC: Not so good for time varying data last time I checked.

Ilmi: might lose access to frame buffer and pixel level manipulation \--

extremely difficult for view dependent or image-based approach

JohnS: It will add greatly to the complexity of this system. It also may
get in the way of novel rendering methods like Image-based methods.

Wes: Back to the scene graph issue - what you allow for is composition
of streams

of data into a renderer. Since view position information is supported as
a

first class DS/DM citizen (right?) it becomes possible to compose a

rendering session that is driven by an external source.

Nearly all renderers use scene graph concepts - resistance is futile!
The

weak spot in this discussion concerns streaming. Since scene graphs
systems

presume some notion of static data, the streaming notion poses some
problems.

They can be surmounted by adding some smarts to the rendering and the

data streaming - send over some bounding box info to start with, then
allow

the streaming to happen at will. The renderer could either then not
render

that tree branch until transmission is complete, or it could go ahead
and

render whatever is in there at the time. Middle ground could be achieved

with progressive transmission, so long as there are \"markers\" that
signal

the completion of a finished chunk of data to be rendered.

Some people\'s \"complaints\" about scene graphs stem from bad designs

and bad implementations. A \"scene graph system\" is supposed to be

an infrastructure for storing scene data and rendering. That ought to

include support for image-based methods, even though at first blush

it seems nonsensical to talk about buckets-o-triangles in the same

breath as normal maps. All interactive rendering systems are
fundamentally

created equally in terms of intent & design. The implementation varies.

Among the top items in the \"common\" list is the need to store data,
the

need to specify a viewpoint, and the need to propogate transformation

information. Beyond that, it\'s merely an implementation issue.

I caution against spending too much time worrying about how scene graphs

fit into DiVA because the issue is largely a red herring.

> \* What about Postscript, PDF and other scale-free output methods for
> publication quality graphics? Are pixmaps sufficient?

Randy: Gotta make nice graphs. Pixmaps will not suffice.

JohnC: Well what are we trying to provide, an environment for analysis
or

producing images for publications? The latter can be done as a post

process and should not, IMHO, be a focus of DIVA.

JohnS: Pixmaps are insufficient. Our data analysis infrastructure has
been moving rapidly away from scale-free methods and rapidly towards
pixel-based methods. I don\'t know how to stop this slide or if we are
poised to address this issue as we look at this component model.

Wes: Gotta have vector graphics.

> In a distributed environment, we need to create a rendering subsystem
> that can flexibly switch between drawing to a client application by
> sending images, sending geometry, or sending geometry fragments
> (image-based rendering)? How do we do that?

Randy: See the Chromium approach. This is actually more easily done than

one might think. Define an image \"fragment\" and augment the rendering

pipeline to handle it (ref: PICA and Chromium).

JohnC: Use Cr

Jim: I would think this could be achieved by a sophisticated data
communication

protocol - one that encodes the type of data in the stream, say, using
XML

or some such thingy.

Wes: Again, one size doesn\'t fit all. These seem to be logically
different components.

Note: So we are going to define the OpenGL/Cr API as a "port" interface
in CCA? All of GL will go through RMI?

> \* Please describe some rendering models that you would like to see
> supported (ie. view-dependent update, progressive update) and how they
> would adjust dynamically do changing objective functions (optimize for
> fastest framerate, or fastest update on geometry change, or varying
> workloads and resource constraints).

Randy: See the TeraScale browser system.

JohnC: Not sold on view dependent update as worthwhile, but progressive
updates

can be hugely helpful. Question is do you accomplish this by adding

support in the renderer or back it up the pipeline to the raw data?

JohnS: I see this as the role for the framework. It also points to the
need to have performance models and performance monitoring built in to
every component so that the framework has sufficient information to make
effective pipeline deployment decisions in response to performance
constraints. It also points to the fact that at some level in this
component architecture, component placement decisions must be entirely
abstract (but such a capability is futureware).

So in the short-term its important to design components with effective
interfaces for collecting performance data and representing either
analytic or historical-based models of that data. This is a necessary
baseline to get to the point that a framework could use such data to
make intelligent deployment/configuration decisions for a distributed
visualization system.

Wes: The scene graph treatise (above) covers most of what I have to say
for now.

# Note: Randy gets to present the features of the TeraScale browser system

> \* Are there any good examples of such a system?

Randy: None that are ideal :), but they are not difficult to build.

JohnC: Yes, Kitware\'s not-for-free volume renderer (volren?). I does a
nice job

with handling progressive updates. This is mostly handled by the GUI but

places some obvious requirements on the underlying rendering/viz

component.

JohnS: No. That's why we are here.

Wes: I know of a couple of good scene graphs that can form the basis for
renderers.

> What is the role of non-polygonal methods for rendering (ie. shaders)?

-   Are you using any of the latest gaming features of commodity cards
    in your visualization systems today?

JohnC: Yup, we\'ve off loaded a couple of algorithms from the CPU.

We just have some very simple, one-off applications that off-load

computation from the cpu to gpu. For example, we have a 2D Image Based

Flow Visualization algorithm that exploits vertex programmability to do

white noise advection. Developing this type of application within

any Diva framework I\'ve envisioned would really push the limits of

anything we\'ve discussed.

JohnS: I\'d like to know if anyone is using shader hardware. I don\'t
know much about it myself, but it points out that we need to plan for
non-polygon-based visualization methods. Its not clear to me how to
approach this yet.

\* Do you see this changing in the future? (how?)

Randy: This is a big problem area. Shaders are difficult to
combine/pipeline.

We are using this stuff now and I do not see it getting much easier

(hlsl does not fix it). At some point, I believe that non-polygon

methods will become more common that polygon methods (about 3-4 years?).

Poylgons are a major bottleneck on current gfx cards as they limit

parallelism. I\'m not sure what the fix will be but it will still be

called OpenGL :).

JohnC: The biggest issue is portability, but things are looking up with
OpenGL

2.0 efforts, etc.

Wes: We\'ve invited Ilmi Yoon to the next workshop. She represents the
IBR

community. I am very keen to see us take advantage of IBR techniques as
well

as our traditional polygon engines, perhaps combining them in
interesting

ways to realize powerful new systems.

Note: Do scene graphs somewhat address portability issues via further
abstraction of the rendering procedure?

> 5\) Presentation=========================
>
> It will be necessary to separate the visualization back-end from the
> presentation interface. For instance, you may want to have the same
> back-end driven by entirely different control-panels/GUIs and
> displayed in different display devices (a CAVE vs. a desktop machine).
> Such separation is also useful when you want to provide different
> implementations of the user-interface depending on the targeted user
> community. For instance, visualization experts might desire a
> dataflow-like interface for composing visualization workflows whereas
> a scientists might desire a domain-specific dash-board like interface
> that implements a specific workflow. Both users should be able to
> share the same back-end components and implementation even though the
> user interface differs considerably.
>
> How do different presentation devices affect the component model?

Jim: Not. The display device only affects resolution or bandwidth
required.

This could be parameterized in the component invocations APIs, but

should not otherwise change an individual component.

If you want a \"multiplexer\" to share a massive data stream with a
powerwall

> and a PDA, then the \"multiplexer component\" implementation handles
> that\...
>
> \* Do different display devices require completely different user
> interface paradigms? If so, then we must define a clear separation
> between the GUI description and the components performing the back-end
> computations. If not, then is there a common language to describe user
> interfaces that can be used across platforms?

Randy: I think they do (e.g. immersion).

Jim: No. Different GUIs should all map to some common framework
command/control

interface. The same functions will ultimately get executed, just from
buttons

with different labels or appl-specific short-cuts\... The UIs should all
be

independent, but talk the same protocol to the framework.

Yuk (with regard to creating separation between GUI and component
description)

JohnS: Systems that attempt to use the same GUI paradigm across
different presentation media have always been terrible in my opinion. I
strongly believe that each presentation medium requires a GUI design
that is specific to that particular medium. This imposes a strong
requirement that our compute pipeline for a given component architecture
be strictly separated from the GUI that controls the parameters and
presents the visual output of that pipeline. OGSA/WSDL has been proposed
as one way to define that interface, but it is extremely complex to use.
One could use CCA to represent the GUI handles, but that might be
equally complex. Others have simply customized ways to use XML
descriptions of their external GUI interface handles for their
components. The latter seems much simpler to deal with, but is it
general enough?

> \* Do different display modalities require completely different
> component/algorithm implementations for the back-end compute engine?
>
> (what do we do about that??)

Randy: They can (e.g. holography), but I do not see a big problem there.

Push the representation through an abstraction (not a layer).

Jim: Algorithm maybe, component no. This could fall into the venue of
the

different execution-model-specific frameworks and/or their bridging\...

I dunno.

JohnS: I think there is a lot of opportunity to share the back-end
compute engines across different display modalities. There are some
cases where a developer would be inclined to implement things like an
isosurfacer differently for a CAVE environment just to keep the
framerates up high-enought to maintain your sense of immersion. However,
I think of those as edge-cases.

What Presentation modalities do you feel are important and what do you
consider the most important.

> \* Desktop graphics (native applications on Windows, on Macs)

Randy: #1 (by a fair margin)

JohnC: This is numero uno by a HUGE margin

Jim: MUST

JohnS: #1

Wes: Yes, most important, will never go away.

> \* Graphics access via Virtual Machines like Java?

Randy: #5

JohnC: Not important

Jim: Ha ha ha ha...

JohnS: #5

Wes: If it works on desktops, it will work in these environments.

> \* CAVEs, Immersadesks, and other VR devices

Randy: #4

JohnC: Not important

Jim: Must

JohnS: #4

Wes: Second to workstations. With evolution of Chromium, DMX and the
nascent

PICA stuff, I would expect that desktop tools would port transparently

to these devices.

> \* Ultra-high-res/Tiled display devices?

Randy: #3 - note that tiling applies to desktop systems as well, not

necessarily high-pixel count displays.

JohnC: Moderately important

Jim: Must

JohnS: #3 : the next tiled display may well be your next \*desktop\*
display, but quite yet.

> \* Web-based applications?

Randy: #2

JohnC: Well, maybe.

Jim: Probably a good idea. Someone always asks for this\... :-Q

JohnS: #2

> What abstractions do you think should be employed to separate the
> presentation interface from the back-end compute engine?

Jim: Some sort of general protocol descriptor, like XML\...? Nuthin
fancy.

> \* Should we be using CCA to define the communication between GUI and
> compute engine or should we be using software infrastructure that was
> designed specifically for that space? (ie. WSDL, OGSA, or CORBA?)

Randy: No strong opinion.

Jim: The CCA doesn\'t do such communication per se. Messaging between or
in/out

of frameworks is always \"out of band\" relative to CCA port
invocations.

If the specific framework impl wants to shove out data on some wire,

then it\'s hidden below the API level\...

I would think that WSDL/SOAP would be O.K. for low-bandwidth uses.

JohnS: I think I addressed this earlier. We can do this all in CCA, but
is that the right thing to do? I know this is an implementation issue,
but is a strong part of our agreement on methods to implement our
components (or define component boundaries).

Wes: (I this as similar to rendering in VM's like Java in many
respects).

Always sounds nice, but have

yet to see much fruit in this area. The potential importance/relevance
is great.

The browser makes a nice UI engine, but I wouldn\'t trust it to do
\"real\"

rendering.

> \* How do such control interfaces work with parallel applications?
>
> Should the parallel application have a single process that manages the
> control interface and broadcasts to all nodes or should the control
> interface treat all application processes within a given component as
> peers?

Randy: Consider DMX, by default, single w/broadcast, but it supports

backend bypass\...

Jim: I vote for the \"single process that manages the control interface
and

broadcasts to all nodes\" (or the variation above, where one of the

parallel tasks forwards to the rest internally :-). The latter is

not scalable.

BTW, you can\'t have \"application processes within a\... component\".

What does that even mean?

Usually, an application \"process\" consists of a collection of one or

more components that have been composed with some specific
connectivity\...

JohnS: This requires more discussion, but reliable broadcast methods
have many problems related to event skewing and MPI-like point-to-point
emulation of the broadcast suffers from scalability problems. We need to
collect design patterns for the control interface and either compete
them against one-another or find a way to support them all by design.
This is clearly an implementation issue, but will leak in to our
abstract component design decisions. Clearly we want a single thread of
control to efficiently deliver events to massively parallel back-end
components. That is a \*must\* requirement.

Note: That paradigm (one component-chain per process) doesn't offer you
much opportunity for encapsulating complex parallel communication
patterns.

> 6\) Basic Deployment/Development Environment Issues============
>
> One of the goals of the distributed visualization architecture is
> seamless operation on the Grid \-- distributed/heterogeneous
> collections of machines. However, it is quite difficult to realize
> such a vision without some consideration of deployment/portability
> issues. This question also touches on issues related to the
> development environment and what kinds of development methods should
> be supported.
>
> What languages do you use for core vis algorithms and frameworks.
>
> \* for the numerically intensive parts of vis algorithms

Randy: C/C++ (a tiny amount of Fortran)

JohnC: C/C++

Jim: C/C++... Fortran/F90 for numerically intensive parts.

JohnS: C/C++/Fortran

Wes: C/C++

> \* for the glue that connects your vis algorithms together into an
> application?

Randy: C/C++

JohnC: C/C++, Tcl, Python

Jim: C/C++

JohnS: C++/C/Java, but want to get into some Python (it is said to have
better numerics than Java)

Wes: C/C++

> \* How aggressively do you use language-specific features like C++
> templates?

Randy: Not very, but they are used.

JohnC: Not at all. Too scary.

Jim: RUN AWAYYYY!!! These are not consistent across o.s./arch/compiler
yet.

Maybe someday\...

JohnS: I avoid them due to portability and compiler maturity issues.

Wes: Beond vanilla classes, not at all.

> \* is Fortran important to you? Is it important that a framework
> support it seamlessly?

Randy: Pretty important, but at least \"standardly\" enhanced F77 should
be simple :).

JohnC: Nope

Jim: Fortran is crucial for many application scientists. It is not
directly

useful for the tools I build.

But if you want to ever integrate application code components directly

into a viz framework, then you better not preclude this\... (or
Babel\...)

JohnS: Yes, absolutely. It needn\'t be full fledged F90 support, but
certainly f77 with some f90 extensions.

Wes: No. Fortran can be wrapped inside something sane.

Note: It is perhaps incumbent on us to support Fortran. We would
eventually like buy-in from domain scientists to provide some analysis
components that are interesting for them. Lack of Fortran bindings for
VTK was a major issue for some participants in the Vis Greenbook
workshop.

> \* Do you see other languages becoming important for visualization
> (ie. Python, UPC, or even BASIC?)

Randy: Python is big for us.

JohnC: Python, mostly because the direction of numerical python.

Jim: Nope.

JohnS: Python

> What platforms are used for data analysis/visualization?
>
> \* What do you and your target users depend on to display results?
> (ie. Windows, Linux, SGI, Sun etc..)

Randy: Linux, SGI, Sun, Windows, MacOS in that order

JohnC: All the above, primarily lintel an windoze though.

Jim: All of the above (not so much Sun anymore...)

JohnS: Linux, MacOS-X(BSD), Windows

Wes: For rendering, OpenGL engines.

> \* What kinds of presentation devices are employed (desktops,
> portables, handhelds, CAVEs, Access Grids, WebPages/Collaboratories)
> and what is their relative importance to active users.

Randy: Remote desktops and laptops. Very important

JohnC: desktops, tiled displays, AG

Jim: All but handhelds are important, mostly desktops, CAVEs/hi-res and
AG,

in decreasing order.

JohnS: Desktop and laptops are most important. Web, AG, and CAVE are of
lesser importance (but still important).

Wes: Workstations are most important.

\* What is the relative importants of these various presentation methods
from a research standpoint?

Randy: PowerPoint :)?

JohnC: The desktop is where the users live.

Jim: CAVEs/hi-res and AG are worthwhile research areas. The rest can be

weaved in or incorporated more easily.

\* Do you see other up-and-coming visualization platforms in the future?

Randy: Tablets & set-top boxes.

JohnC: I don\'t see SMP graphics boxes going away as quickly as some
might.

Jim: Yes, but I haven\'t figured out where exactly to stick the chip
behind

my ear for the virtual holodeck equipment\... :)

JohnS: Tablet PCs and desktop-scale Tiled display devices.

> Tell us how you deal with the issue of versioning and library
> dependencies for software deployment.
>
> \* For source code distributions, do you bundle builds of all related
> libraries with each software release (ie. bundle HDF5 and FLTK source
> with each release).

Randy: For many libs, yes.

JohnC: Sometimes, depending on the stability of the libraries.

Jim: CVS for control of versioning.

For bundling of libraries: No, but provide web links or separate copies
of dependent distributions

next to our software on the web site\...

Too ugly to include everything in one big bundle, and not as efficient

as letting the user download just what they need. (As long as everything

you need is centrally located or accessible\...)

JohnS: Every time I fail to bundle dependent libraries, it has been a
disaster. So it seems that packaging dependent libraries with any
software release is a \*must\*.

Wes: Oddly enough, I do bundling like this for some of my projects. I
think people

appreciate it.

> \* What methods are employed to support platform independent builds
> (cmake, imake, autoconf). What are the benefits and problems with this
> approach.

Randy: gmake based makefiles.

JohnC: I\'ve used all, developed my own, and like none. Maybe we can do
better.

I think something based around gmake might have the best potential.

Jim: Mostly autoconf so far. My student thinks automake and libtools is
\"cool\"

but we haven\'t used them yet\...

JohnS: I depend on conditional statements in gmake-based makefiles to
auto-select between flags for different architectures. This is not
sufficiently sophisticated for most release engineering though. I have
dabbled with autoconf, but it is not a silver bullet (neither was
imake). I do not understand the practical benefits of \'cmake\'.

Wes: I hate Imake, but used it extensively for a long time with LBL\'s
AVS modules.

I think it still works. Nobody I know can figure out how autoconf works.
I

personally tend to have different makefiles, particularly when doing
code

that is supposed to build on Win32 as well as Unix/Linux systems.

> \* For binaries, have you have issues with different versions of
> libraries (ie. GLIBC problems on Linux and different JVM
> implemetnations/version for Java). Can you tell us about any
> sophisticated packaging methods that address some of these problems
> (RPM need not apply)

Randy: No real problems other that GLIBC problems. We do tend to ship
static

for several libs. Motif used to be a problem on Linux (LessTiff vs

OpenMotif).

Jim: Just say no. Open Source is the way to go, with a small set of
\"common\"

binaries just for yuks. Most times the binaries won\'t work with the

specific run-time libs anyway\...

JohnS: Building statically has been necessary in a lot of cases, but
creates gigantic executables. In the case of JVM\'s, the problems with
the ever-changing Java platform have driven me away from employing Java
as a development platform.

Wes: I tend to just do source, rather than binaries, to avoid this whole
morass.

OTOH, as a consumer, I prefer RPMs so that I don\'t have to build it. I
want

my ice toasted, please.

> \* How do you handle multiplatform builds?

Randy: cron jobs on multiple platforms, directly from CVS repos. Entire

environment can be built from CVS repo info (or cached).

JohnC: The brute force, not so smart way. The VTK model is worth looking
at.

Jim: Autoconf, shared source tree, with arch-specific subdirs for object
files,

libs and executables.

JohnS: \* Conservative, lowest-common denominator coding practices.

\* execute \'uname\' at the top of a gnu makefile to select an
appropriate set of build options for sourcecode building. Inside of the
code, must use the CPP to code around platoform dependencies.

> How do you (or would you) provide abstractions that hide the locality
> of various components of your visualization/data analysis application?

Jim: I would use \"proxy\" components that use out-of-band communication
to

forward invocations and data to the actual component implementation.

-   Does anyone have ample experience with CORBA, OGSA, DCOM, .NET, RPC?
    Please comment on advantages/problems of these technologies.

Jim: Nope

JohnS: Nope

\* Do web/grid services come into play here?

Randy: Not usually an issue for us.

Jim: Yuck, I hope not...

JohnS: As these web-based scientific collaboratory efforts gather
momentum, web-based data analysis tools have become increasingly
important. I think the motivation is largely driven by deployment issues
when supporting a very heterogeneous/multi-institutional user base. It
reduces the deployment variables when your target is a specific
web-server environment, but you pay a price in that the user-interface
is considerably less advanced. This cost is mitigated somewhat if the
data analysis performed is very domain-specific and customized for the
particular collaboratory community. So its a poor choice for
general-purpose visualization tools, but if the workflow is
well-established among the collaborators, then the weakness of the
web-based user-interface options is not as much of a problem.

> 7\) Collaboration ==========================
>
> If you are interested in \"collaborative appllications\" please define
> the term \"collaborative\". Perhaps provide examples of collaborative
> application paradigms.

Randy: Meeting Maker? :) :) (I\'m getting tired).

Jim: \"Collaborative\" is 2 or more geographically/remote teams, sharing
one

common viz environment, with shared control and full telepresence.

(Note: by this definition, \"collaborative\" does not yet exist\... :-)

JohnS: Despite years of dabbling in "collaborative applications," I'm
still not sure if I (or anyone) really knows what "collaborative" is in
a strict sense.

Wes: The term \"collaboration\" is one of the most overused, misused and
abused

terms in the English language. There is a huge disconnect between what

many users want/need, and what seems to be an overemphasis upon
collaborative

technologies. For this particular project, collaboration (ought to)
mean:

being able to share software components; and some level of confidence
that

\"DiVA-compliant\" components in fact do interoperate. For the sake of

discussion, let\'s call this type of collaboration \"interoperability.\"

For the other forms of \"collaboration,\" care must be taken to define
what

they are, whether they are useful, etc. If you\'re talking about
multiple

persons seeing the same interactive renderer output, and each person
being

able to do some interactive transformation, let\'s call that form of

collaboration \"MI\" (multiperson-interactive).

I recall hearing some discussion about the relationship between the AG

and DiVA. From my perspective, the AG ought to provide support to allow

any application to run in a \"MI mode\" With this perspective,

there isn\'t really much to talk about in terms of fundamental DiVA

design wrt \"MI.\"

Is collaboration a feature that exists at an application level or are
there key requirements for collaborative applications that necessitate
component-level support?

-   Should collaborative infrastructure be incorporated as a core
    feature of very component?

JohnC: Does it need to be incorporated in all components? What kind of
collab

support is needed? Permitting session logging and geographically

separated, simultaneous, users would go a long way to providing for

collab needs and would seem to only impact the GUI and perhaps renderer.

Jim: Collaboration should exist \*above\* the application level, either
outside

the specific framework or as part of the framework \"bridging\"
technology.

JohnS: No. I hope that support for collaborative applications can be
provided via supplemental components.

Wes: I don\'t know what \"collaborative infrastructure\" means. Given
that my position

(above), \"MI\" is more of a framework thing, and not a component thing.

This seems to be the most realistic approach to \"MI.\"

*Note: I'm not sure how to interpret this answer. Is this a "framework
issue" or a "component issue" or is it totally outside of the
application? So we retrofit applications to be "collaborative" from the
outside rather than designing apps or the frameworks that implement them
to support collaboration "requirements" as a fundamental feature of the
technology?*

-   Can any conceivable collaborative requirement be satisfied using a
    separate set of modules that specifically manage distribution of
    events and data in collaborative applications?

Jim: I dunno, I doubt it.

JohnS: That is what I hope.

-   How is the collaborative application presented? Does the application
    only need to be collaborative sometimes?

Jim: Yes, collaboration should be flexible and on demand as needed -
like

dialing out on the speakerphone while in the middle of a meeting\...

JohnS: This is probably true. You probably want to be able to have tools
that were effectively standalone that can join into a collaborative
space on demand.

-   Where does performance come in to play? Does the visualization
    system or underlying libraries need to be performance-aware?

(i.e. I\'m doing a given task and I need a framerate of X for it to be
useful using my current compute resources), network aware (i.e. the
system is starving for data and must respond by adding an alternate
stream or redeploying the pipeline). Are these considerations
implemented at the component level, framework level, or are they
entirely out-of-scope for our consideration?

Jim: There likely will need to be \"hooks\" to specify performance
requirements,

like \"quality of service\". This should perhaps be incorporated as part

of the individual component APIs, or at least metered by the
frameworks\...

It would be wise to specify the frame rate requirement, perhaps
interactively

depending on the venue\... e.g. in interactive collaboration scenarios
you\'d

rather drop some frames consistently than stall completely or in
bursts\...

This sounds like futureware to me - an intelligent network protocol
layer\...

beyond our scope for sure!

These issues should be dealt with mostly at the framework level, if at
all.

I think they\'re mostly out-of-scope for the first incarnation\...

JohnS: Yes. The whole collaboration experience will fall apart if you
cannot impose some constraints on quality of service or react
appropriately to service limitations. Its a big problem, but I hope the
solution does not need to be a fundamental feature of the baseline
component design.

Wes: The MI-aware framework collects and uses performance data generated
by

components to make decisions about how to tune/optimize visualization

pipeline performance (the pipeline consists of a bunch of components).

If some of the other issues I\'ve raised are addressed (e.g.,
time-limited

execution, partial processing, incremental processing, etc), then the

performance issues raised within the context of MI come \"for free\".

*Note: The issue here is that if anyone thinks this should be done at
anything other than the framework (or even outside of framework) level,
then it could be very disruptive to our design process if we develop
first for single-user operation and then later attempt to make
"collaborative services" a requirement. Implementing this at the
"framework level" is again a high-price-for-admission. If there is any
way to support this at a component level, it would enable people working
on collaborative extensions to share better with people who have
different aims for their framework. I don't consider it a benefit to
have one "framework" per use-case as has been the practice in many
aspects of CCA. It will just continue the balkanization of our
development efforts.*