The Engines that Drive Collaboration
Rick Stevens
Mathematics and Computer Science Division
Argonne National Laboratory
Abstract
The emergence of virtual reality technology has introduced an exciting new approach to scientificresearch and development. Virtual reality environments such as the CAVE (Cave Automatic VirtualEnvironment) are allowing researchers to conduct three-dimensional simulations---to go "inside"
experimental data and interact with results computed on high-performance supercomputers. Current
applications include drug design, simulation of the casting process for car and aircraft parts, andmodeling of combustion within commercial boilers. In the future, collaborative virtual laboratories may
enable distributed scientific research groups to share unique large-scale experimental facilities such as
accelerators, electron microscopes, and light sources. Using emerging high-speed networks like thosebeing developed as part of the I-WAY (International Wide Area Network) project, researchers in
different locations will be able to collaborate naturally and effectively over rich virtual environments that
support audio, video, and gesture capabilities. Through telescience technology, researchers also may
remotely control operations in deep-sea exploratory submarines, space shuttles, and nuclear reactors.
Virtual environments present significant technological challenges, but the rewards are clear: new insight;
faster understanding of complex, multidimensional phenomena; and increased scientific productivity.
Noteworthy examples are the I-WAY project, which focuses on mechanisms to link supercomputers andadvanced visualization environments via asynchronous transfer mode; and the LabSpace project,
exploring electronic virtual laboratories to support remote operation and control of scientific experiments.
(The author did not provide a formal written version of his paper but instead made available thetransparencies from his talk.)

The Engines that Drive
Collaboration
Rick Stevens
Mathematics and Computer Science
Division
Argonne National Laboratory
http://www.mcs.anl.gov/people/steve
nsstevens@mcs.anl.gov

What is the talk about..
• Long term vision for collaborativeenvironments• Computing and networking technology• Research projects and directions• Connections to the I-WAY project• Open problems and Invitations forcollaboration

Page 1Roadmap for Research Programs to Support theGlobal Information Infrastructure
Collaborative Modelingtoday’s Grand Challenges

Algorithms• physical models• matrix methods• spectral methods• reactive flows• molecular models• electronic structure• structural mechanics• n-body methods• optimization

Architectures

ToolsNetworks• compilers• debuggers• auto parallel pre-X• event tracing• visualization• algorithm
prototyping• libraries (MPI, p4)• “tool kits”• templates• workstations• clusters• single MPP• vector computers• simple distributed sys• largely single domain• strong on Flops• weak on I/O• small data systems

Languages• ethernet (LANs)• FDDI (LANs)• HIPPI (sub-LANS)• FCS (not yet)
• ATM (not yet)• fiber infrastructure• long haul T1/T3• experimental OC-48• workstation peers

Applications• Fortran 77•C• C++• HPF (not yet)
• Fortran 90 (not yet)
• Fortran M (not yet)
• PCN
• Linda• extensions vs new• extensions of vector
codes• physical models• few non-science• few engineering• mostly modeling notdesign• little linkage to exp• batch still prime mode

Collaborative Analysis and Design
distributed shared resources

Algorithms• caching, browsing• MDA, MDO• statistics• comparative analysis• interactions• CAD linkage tomodels• database linkage• compression• automatic filters• neural networks

Architectures• workstation aggregates• MPP aggregates• heterogeneous nodes• cooperative systems• mobile nodes• database linkages• multimedia support• archival storage• data systems• I/O intensiveTools• multi-user AVS• multi-user MMA• hetero MATLAB• linkage CAD/MMA• linkage CAD/MPP• linkage CAD/DB• linkage CAD/MDA• multisite above• penbased interfaces

Networks• gigabits to the desk
• national filesystem• ATM combo• nonmetered• bandwidth on demand• high peak / low mean
• aggregate for blasting• standard HW/SW• blasting protocol

Languages• integrated analysis
and CAD languages• MDO extensions• hetero parallel• fast MATLAB• AVS like extensions• OBJECT support• implicit I/O support• DB integration• analysis objects

Rick Stevens, Mathematics and Computer Science Division, Argonne National Laboratory

Applications• manufacturing ofcomplex systems• analysis of 3ary
science problems• rigorous analysis ofcomputational exp• simulation intensive
engineeringVirtual Reality andMultimedia
multiviewer interactive
distributed deployment

Algorithms

ArchitecturesTools• projective geometry• rendering• kinematics• linkage to physics• new paradigms forvisualization of data
• interactions• compression• indexing• data movement• CAVEn• CAVE++• HD-CAVE• HD-Goggles• body suit I/O• retinal imagers• 3-D projection• holographic display• holographic input• mobile links• 3-D AVS
• 3-D highend MMA• multisite CAVE env• 3-D tcl scripting• CAVE/ MOSAIC• Goggleland• connections• linkages• pad based kibitzing• scalabilityNetworks• CAVE-to-CAVE• low latency sync lines• scaling up/down• 10’s Gb each site
• fanin/fanout problem• GB to each user• mobile head to head
• LAN-MAN-WAN

Languages

Applications• 3-D and VR extension• VR objects• parallel objects• parallel interacting
agents• coupling to world DBs• visual programming• visual tuning• pen based sketches• 3-D story boards/lang• medical imaging• remote exploration• database mining• education• telecollaborations• 3-D data immersion• micro manufacturing• entertainment• travel reduction

Telepresence
remote manipulationand monitoring
remote “participation”

Algorithms• autonomous systems• solids models• planning, interactions• navigation• multimedia fusion• sensor fusion• human/computer int• compression• geometry reasoning• vision and mages

Architectures• gigabits network• hetero drivers• MPP• workstation• realtime links• multi-sensor• robotic support• manipulators• nanobots• mobile support

Tools• protocols• sockets for multisense• SVP simple video pro• roboKIT interfaces• stationary video interf• voice/vid/window• manipulator KIT• multiview protocol• DB interfaces• agent interfaces

Networks• gigabits/human-rin• short range radio• transparent LB/radio• digital video• low latency ATM• radio ATM interfaces• multimedia mix• multisense mix• network compression

Languages• scripting for RINs
• manipulationscripts• programming byexample• multisense objects• multisense extensions• linkage to multisense• databases• autologing of events• autolog I/O library

Rick Stevens, Mathematics and Computer Science Division, Argonne National LaboratoryApplications• monitoring r activities• r manufacturing• r assembly• r exploration• r experiment manip• r inspections• semi auto develop• r piloting• toysWorkthink Trends• Virtualization of organizations• Merging of work/life/play/home/school• Work as conversation/communication• Dominance of new/multidisciplinary problems• Dominance of consumer products/tech drivers• Self-assembly (business, research, education,recreation)
What is the right infrastructure to support futurework?

How would life bedifferent if travel was freeand instantaneous ?

Page 3Hmmm ?
• You would live where you want to.
• You and your colleagues would meet in interesting places.
• You would leave your things where you need or use them.
• You would need to “arrange” to meet people in certain places atcertain times.
• You would need mechanisms to “locate” people and things.
• Security would be problematic. (see Tales of Known Space)• You would develop habitual hangout places, most likely to findfriends and family.
• You would adjust. (with apologies to John Gorka)

The Need for PersistentElectronic Spaces••••Adding the concept of Persistent Electronic Spaces
to the current suite of computer supported collaborative
work tools can provide the virtual equivalent of instantand (almost) free travel.
Persistence is needed to build electronic communities
Persistence is needed to map the real world to virtualenvironments
Persistence is needed to lead us away from the phone
call model for collaborations towards the “Cafe” model
Persistence is needed to improve navigation anddiscovery processes for the GII

Page 4

Expanded - Work/Product Cycle
conceivealternatives

designengineer

critiquebuildevaluate
usegoal:commit

reduce the cycle timeby making mistakes fastermarket

Virtual Organizations:instant teams
spontaneous workgroups

••
•
•••
•

Have no particular physical location
Can have mission oriented lifetimes
Can have non-hiearchical structure
Require awareness and loyalty of members
Require a variety of communications support
Require sharable resources (data, machines, etc.)
Require mechanisms for organizational navigationand discovery• Require resource allocation mechanisms• Require ability to determine membership status

Page 5

What do we Mean when we say
Virtual Environments?
• 3D real time Graphics• Point of view rendering• Wide field of view (immersive)• High resolution• Audio and other sense modalities (touch, temp,smell, etc.)• Responsive model interactions (low lag)
• Collaborative (i.e. ability to interact with otherpeople & VR at the same time)

Collaborative Environment Scale
Metaphors
Pers onal Des kt op

Des kRoomwrist wat ch t v
earrings
st ereopocket books cd-rom
pens
sket ch padeyeglasses PC
clot hingkeyboard
cell phonemouseheadset st elephone

draf t ing t able
sewing machpianot able saw
cookt opsinklibrary
movie housedark room church
kit chenst ore
wood shop mallart st udio sport s arena
garageconcert hall

PDA

MediaSt at ion Immers aDes k CAVE

Page 6

Theat er

Dis playWall

Continuum of Collaborative
Environment Display
Technologies
Individuals

Personal
Information
Infrastructure

workgroups

collaborations

institutions

Desktop
Collaborative ImmersaDesk
CAVE
Environment I-Desk
Automatic
VirtualEnvironment

Display WallPowerWall
NII/GII Wall

Personal• Lightweight• Wireless• Wearable• Always available• Audio dominate• Keyboardless• $80-$800 price point..

Page 7

Individuals

workgroups

collaborations

institutions

Desktop• Interactive video• Multichannel audio• Text, 2 and 3D graphics• Flexible drawing surfaces• Keyboards/mouse• Ambient and focused media• Multiscreen• $8K price point

Individuals

workgroups

collaborations

institutions

DeskIndividuals• 3D immersion/stereo display• User tracking• One high-resolution screen• Seated or standing• Drafting table form factor• Stereo Audio• 1-3 people• $80K price point

Page 8workgroups

collaborations

institutions

RoomIndividuals• 3D immersion• 1 user tracking• Standing with limited motion• 4 high-resolution walls• 1-8 people• Quadraphonic Audio• $800K price point

workgroups

collaborations

institutions

TheaterIndividuals• Limited 3D Stereo Display• Large format• (NxM high-res screens)• Multichannel Audio• 10-100’s people• $800K+ price point

Page 9workgroups

collaborations

institutions

Wide Area
Digital Fabric

I-WAY Testbed Alpha’
EVL
CAVE

ArgonneCAVE
ArgonneIBM SP2
ATM/IP basedNetworkNCSAPowerChallenge

Page 10EVL
ImmersaDesksNCSACAVE

Integrating Virtual Environments
and Massively Parallel Computing• Scalable interconnect of “rendering engines” tosimulations• Low latency network interface, WAN latency
management• Mechanisms for sharing models (simulation vs.
rendering)• Support for recording/playback• Support of video data types• New programming environments (openinventor++)Argonne CAVE Research Environment Architecture
128 node IBM SP-2
RS6K

Ethernet
ATM OC-3c
IBMVulcan
Switch

RS6K

RS6K

NSC HIPPI
SwitchHIPPIIBM HSC (TB2)
RS6K

Silicon Graphics ONYX

M8000

SGI Serial

HIPPI

SGI Serial

ATMIBM PC

Left Wall

M8000

Reality Engine

WandHeadTracker
Reality EngineONYX Shared Memory

Front Wall

Ethernet

R4400
R4400
R4400

M8000

Reality Engine
R4400

CAVE Entrance
R. Stevens, 1995

Page 11Current and Future Application
Bandwidth Requirements
••
•
••
•
•
•

Remote CAD prototyping (50 Mb/s)
Typical 8-way LabSpace session w/archive (100 Mb/s)
Cave to Cave applications (200 Mb/s)
Nanotech design transmission (500 Mb/s)
8-way shared space CAVE session (1 Gb/s)
APS detector data rate (1 Gb/s)
Raw CAVE I/O (8 Gb/s)
DOE Metalab backbone (500*100 Mb/s) ==> 5x10^4 Mb/s==> 50 Gb/s
• Realtime earth system simulation ( 1 Tb/s)• Distributed teraFLOP supercomputing (2 TF ==> 2 Tb/s)

Campus Digital Resource EngineInternalInterconnection
NetworkMedia ServerNetwork

ComputeServer
400 GFlops
400 GB RAM

VirtualReality
Engine8-16 StereoPlanes

DigitalMediaServer
64 In strms
320 Out strms

StorageEngine5 TB disk
100 TB tapePage 12

VR NetworkGeneral
I/O
Network

Core Digital Resource Engines• Cycle Server• I/O - mass store-- persistence for theenvironment• Stream media -- page servers -video/audio/web++/events• Graphics -- geometry-rendering servers -engines for visual immersion• Networks – internal component interconnects --> 1 GB/s/link– local fabric -- shallow networks --> 1 Gb/s/link– wide area -- deep networks --> 1 Gb/s/link

Software Environment• object oriented --> multithreaded --> language
independent mechanisms• integration with communications and security• integration with media stream support(multicast, real-time etc.)• building blocks for collaborations• brokering/proxy capable• multi-paradigm• storage integrated-- performance centric-persistent• dynamically downloadable• peerable server/client/models

Page 13

“Integrating ResearchThemes”
Connections to university
Mathematics andComputer Sciencegtin
ty pu gyali om olol re d c chn
tua ute tevir trib kingdis twornenu
alg mervis orit ical cua hm om
liz spuatitinongmath and cs research programs

Linking
Experimentaland Theoretical Science

Connection totheory
programs

CollaborativeEnvironments

Apex Computing
computationalscience

Connection toexperimental
facilities

Philosophical Goals Behind Voyager
and Avalon• Techno-Empowerment: recognizes the great
disparity in productivity and cognitive skillspresent in groups and organizations. The key point
is to amplify the best and brightest, the problem
solvers and the primary integrating thinkers. Inaddition through technology, uplift the middle andmargins and empower everyone to contribute toorganizational goals.
• Key problems: scalable interactions andcoordination.

Page 14

Classical Modes of Working•
•
•
•
••
•
•
•
•

Seminars (peers)
Classes (students, teacher)Workshops (long duration)
Meetings (short duration)
Conferences (multisession)
Panels/Roundtables
Presentations
Retreats
Hotel Rooms
Breakfast••••••••••

Collaborative Hacking
SoloOne-on-One
Hallways-coffee breaks
Help Desk
Sales calls
WalksLectures
Airplanes
Lunch

Argonne Voyager• A project to develop servers to support thecommunications needs of virtual organizations.
• First effort is a video server (fully symmetricrecord, playback engine).
• Video server will be demonstrated at SC95 as partof the I-WAY activities.
• After SC95 we will be exploring support for awide variety of data types and media streams.
• Scalability is a major goal.

Page 15

SC95 ANL/IBM VideoServer
Logical Diagram

Client Machine
nevotvic

Client MediaApplications

IBM SP2Catalog Server

Tiger SharkMedia ServerdaemonsStream Writer

Stream Reader

Tiger Shark

Tiger Shark

20 SP2 nodes

Virtual Shared Disk
via IBM HPS

8 SP2 nodes
500GB SSA Disk

SC95 ANL/IBM Video Server
Physical Layout
IBM RS/6000
Model 43P or 41T
Analog video in/out
Audio in/out
OC3 ATM

500GB SSA disk

28 x OC3 ATM

Wave ATM Fabric

28-node IBM SP2

8 SSA channels

Page 16Avalon: A One Thousand User
Collaborative Environment• Future research project at Argonne to put a thousandpeople on-line in a shared virtual environment.
• Prototype of an Integrated National Collaboratory.
• Scientific, support and administrative users.
• Based on Voyager, LabSpace, Zipper and I-WAY
inspired technologies.
• Technology Goal: an ATM based 1000 desktop
environment with interactive video, audio, text, web,computation, robotics and database resources.

Collaborative Environments -Telepresence• Provide the user with flexible ways to “be there”• Includes both high-end tools (CAVE-to-CAVE) anddesktop collaborative environments (LabSpace)• I-WAY demonstration of telerobotic “proxy”• Has major potential to change the way people work– telecommuting, telescience– remote control of experiments– distributed collaborative research groups– works for administrative as well as technical users– can reduce costs and reduce staffing levels

Page 17

Distributed Collaborative Environments
Technology• Software and systems to support distributed scientific andengineering collaborations• The “LabSpace” system combines audio, video and data
streams into a “telepresence’ environment to support remote
operation and control of scientific experiments• System will be piloted with users in Materials Science andCAT*s of the Advanced Photon Source• Users will be able to collect and analyze data remotely andinteract naturally with colleagues• This technology can dramatically expand the utilization ofDOE’s ER facilities (e.g electron microscopes, light sourcebeam lines)* Collaborative Access Teams

Elab Server• Text based- multi-user virtual environment• Based on Lambda MOO server technology fromXerox PARC (e.g. BioMOO, AstroVR, InfoPark,WaterfallGlen, JHM, Lambda)• Provides virtual environment “context”• Manages Database of virtual locations, attributes,access mechanisms, users and objects• Extensible and Object Oriented• Scalable and portablePage 18LabSpace
Cave

User

User

User

ModuleModuleModule

User

User

User

User

ModuleModuleModuleArchitecture

User
ModuleModuleOverview
ParallelSupercomputer

InfoServer

Session Broker
Connection(mixer)Data

Elab Server
MultiStream Archiver

Object Oriented

parallel record/playback

Browser withVirtual World

Multiuser VirtualengineCatalog

Database

Environment
Persistent Electronic
Spaces, Places, Tools,

Entry Point

Rooms and Users

Roadmaps
Guide Books
Tours

Stream Data

Stream Data

Stream Data

Session Broker

Connection
Data(mixer)Parallel I/O Environment

User
ModuleUser
ModuleUser
ModuleMicroscope

User

User

ModuleModuleCave

User
User
ModuleModuleUser
Module