****
3.  Advanced Software Technologies

 Today's high performance computing environment differs in three  fundamental ways from theenvironment of 1991 when the HPCC  Program began: 
- The environment includes parallel systems that are significantly more complicated and powerfulthan scalar and vector systems.  One application program can be executed on several computing  systems.  These systems can bescalar or vector or parallel or a  combination thereof; they can range in size from workstations tothe  highest performance systems; they can include special purpose  systems such as workstations forscientific visualization or  expensive remote instruments such as telescopes or microscopes;  they cancome from different vendors; and they can be located at  sites hundreds or thousands of miles apartthat are connected by  high speed networks.  The most demanding application programs are  beginning tobe executed in exactly this mode 
(such applications are  describedbeginning in Section II.6). 
Users access this environment from scientific workstations that  often are critical parts of theenvironment itself -- the workstations  display progress of the execution and results from thecomputations,  and users in turn interact with the execution in real time. 
This contrasts with the 1991 environment in which users commonly  ran their applications on singlescalar or vector computers accessed  via comparatively low speed local area networks and interacted little with the execution, or they ran their applications on their own  workstations. 
Several software developments have made the new environment  possible.  These include the following: 
**Systems Software**
New scalable microkernel operating systems are the product of R&D  during earlier years of the HPCCProgram and have become more  robust, more stable, and more complex.  These operating systems  managedozens to thousands of parallel processors and associated  memory (as well as vector processors insome cases); control I/O to  peripherals such as disk drives, tape drives, printers, and advanced mass storage systems that incorporate robotics; and facilitate  communications with more complicatedand higher speed networks. 
The microkernel basis for HPCC prototype operating systems has  now been accepted by the broadercomputing community.  In the past  year, DEC, Hewlett-Packard, IBM, and Microsoft committed to use microkernel operating systems as the basis of their own offerings.   The overhead involved intransitioning into and out of the  microkernel has received considerable attention in the past two years.  Technology developments include software fault isolation,  co-location, and extensiblemicrokernels. 
**Scalable I/O**
Begun in FY 1994, the Scalable I/O Grand Challenge addresses the  need to complement fast computingsystems with fast input/output  to memory.  Teraflops systems that access terabytes of data will require Gb/s I/O.  By instrumenting I/O intensive Grand Challenge  applications, insights into I/Obehavior are providing opportunities  for I/O performance optimization.  By adding features tooperating  systems, languages, compilers, memory management, interfaces to  mass storage systems, andruntime support, I/O performance  increases of 10-fold to 100-fold are possible.  The initiative is funded by ARPA, DOE, NASA, and NSF, and involves vendors and  researchers from more than 30institutions.  Testbeds include a  512-node Intel Paragon at Caltech's Concurrent Supercomputing Consortium, and a 128-node IBM Scalable POWERparallel (SP) at  DOE's Argonne National Laboratory(ANL).  Products will include an  integrated set of tools and software systems providing scalable I/O capabilities available to the U.S. high performance computing  community, and a scalable I/O benchmarksuite.  One example is the  Pablo software used for performance instrumentation, analysis, and visualization.  Pablo development was funded by ARPA and NSF; it  has been commercialized by Intel. 
http://www.ccsf.caltech.edu/bluebook_96/nsfgcpiom.html
http://www-pablo.cs.uiuc.edu/HPCC.html