# Presentation: 417591

## IMSL Fortran LibraryOverview and Usage Seminarby Visual NumericsLawrence Berkeley National LabMay 19, 2004

- www.vni.com

## Agenda 

- Overview of Visual Numerics, Inc.
- IMSL Fortran 5.0 Overview
- IMSL Fortran 5.0 Standards
- IMSL Fortran Library - HPC
- Supporting IMSL Fortran 5.0
- Example Programs Using IMSL and Fortran 90

## Overview of Visual Numerics, Inc.Tim Leite Director Education Programs

- www.vni.com

## About Visual Numerics

- Established in 1970; privately held
- Over 30 years experience delivering trusted technology solutions for understanding data
- Computational techniques for analyzing data
- quantitative analysis
- Visual techniques for spotting trends / anomalies in data
- qualitative analysis
- Offices worldwide
- USA, UK, France, Germany, Mexico, Japan, Korea, Taiwan
- International distributor network
- Fortune 100 accounts around the globe and well over 500,000 users worldwide

**Notes:**
 - For those of you that are not familiar with Visual Numerics, we will provide a short overview of our company.

 - Independent ISV in business for more than 30 years delivering industry standard technology solutions for analyzing and understanding data.

 - The products from Visual Numerics provide solutions for understanding data.

## Flagship Customers by Industry

## IMSL Fortran 5.0Dr. Edward StewartManager of IMSL Technical Support Engineers

- www.vni.com

## IMSL Overview: Platform History  

- June 1972
- IMSL released for IBM MVS
- First customer was Michigan Tech
- March 1974 - released for UNIVAC
- CDC, Data General were next
- At one time supported 65 platforms
- 1987 - 9.2 to 10.0 (1.0) major release and restructure of IMSL library
- Today we support approximately 20 platforms – market consolidation

## IMSL Fortran Overview (cont) 

- Ease of use features
- Optional arguments implemented throughout the library
- Standard naming conventions for routine names, matrices, variables, parameters (LFTRG, UMING)
- Support for F90 language features
- Extensive use of system environment variables (for error handlers, command line builds, license management, picking up mod files)
- Mod files for parameter list checking at compile time
- Fully backwards compatible
- Single package
- All F77, F90, and parallel processing features

## IMSL Fortran Overview (cont)

- Performance
- OpenMP/SMP support
- Linear Systems, matrix manipulation
- Eigensystem Analysis, Fast Fourier Transforms
- BLAS (Basic Linear Algebra Subprograms)
- Embedded or Vender-supplied (compiler switch)
- ScaLAPACK utilities
- MPI Enabled with MPI-Enhanced interface

**Notes:**
SMP/OpenMP SupportThe IMSL Fortran Library Version 5.0 offers expanded SMP support for a number of parallel processing environments. Computationally intensive algorithms in the areas of linear systems and matrix manipulation, eigensystem analysis, and fast Fourier transforms (FFTs) leverage SMP capabilities on a variety of systems.MPI EnabledIMSL Fortran Library provides a dynamic interface for computing mathematical solutions over a distributed system via Message Passing Interface (MPI). MPI enabled routines offer a simple, reliable user interface.The IMSL Fortran library provides a number of MPI-enabled routines with an MPI-enhanced interface that provides: 
Computational control of the server node. 
Scalability of computational resources. 
Automatic processor prioritization. 
Self-scheduling algorithm to keep processors continuously active. 
Box data type application. 
Computational integrity. 
Dynamic error processing. 
Homogeneous and heterogeneous network functionality. 
Use of descriptive names and generic interfaces 
A suite of testing and benchmark software

## IMSL Fortran Overview (cont)

- The Library uses Fortran 95 features.
- Simple, powerful, and flexible interface
- Short list of required arguments
- Easier to remember nomenclature and syntax
- Full depth and control via optional arguments
- Advanced interface available on all applicable routines formerly in the F77 library
- Maintains full backward compatibility

**Notes:**
NewOptional arguments allow developer to specify variables or leave them out.  If they leave them out, the code will use a default value.  
Shortens code, making it faster to write, easier to track and debug 
Feature from F90 protocol that has been incorporated in all applicable areas
We’ve heard from many customers that they prefer IMSL’s naming conventions and syntax in the industry.  The function names are more natural.  

## IMSL Fortran 5.0 Standards

- www.vni.com

## IMSL Fortran 5.0 Standards

- Library Structure
- Naming Conventions
- Interfaces
- Module Units
- Optional arguments
- Operators
- Generic functions
- IMSL Error Handler system
- EIAT’s (Testing Codes)
- BLAS
- Machine Constants

## IMSL Fortran 5.0 StandardsLibrary Structure

- Routines are usually nested in layers
- Top-level Routine  (Example NEQNF)
- Level-two routines used to allocate workspace and override defaults, ex:  N2QNF
- Level-four routines used to set or change non-default values for the routine, ex: N4QNF Used to set values in IPARAM and RPARAM arrays
- Newer F90 style routine interfaces use optional arguments to set and pass non-default parameter settings

## IMSL Fortran 5.0 Standards Library Structure (cont)

- Newer F90 routines versus older FNL style routines
- F90 has longer more descriptive names
- F90:  LIN_SOL_GEN
- F77:  LSARG
- Optional Argument Interface
- New routines and interfaces written to take advantage of F90 language features (i.e. LDA can be picked up from array size)

## IMSL Fortran 5.0 StandardsLibrary Structure (cont)

- Shared (DLL in Windows) vs. Static
- Shared libraries
- Resolve symbols at run-time
- May make application maintenance easier, don’t have to change out DLL on multiple systems
- Distributed executables are smaller, but DLL must be made available.

## IMSL Fortran 5.0 StandardsLibrary Structure (cont)

- Static libraries
- Resolves symbols at link time
- Executable is complete, but larger
- No DLLs or shared images are necessary
- All applications must be recompiled if change is made to a dependent library such as IMSL

## IMSL Fortran 5.0 StandardsNaming Conventions

- Naming Conventions (e. g. Linear Prog.)
- F77 interfaces - DLPRS and DDLPRS
- F90 Interfaces
- Generic – no prefix (DLPRS)
- s_    single precision (S_DLPRS)
- d_   double precision (D_DLPRS)
- c_   complex, mostly used in newer F90 specific routines
- z_   double complex, mostly used in newer F90 specific routines

## IMSL Fortran 5.0 Standards Interfaces: Mod Files

- Used for parameter list checking
- Extensive use of Module Files (Mod files):
- F90 routines - Use imsl_libraries
- F77 routines  - Use numerical_libraries
- First letter of routine – Use numerical_libraries_n
- Individual routines (F90 interface) - Use neqnf_int

## IMSL Fortran 5.0 Standards Interfaces: Optional Arguments

- Optional Arguments
- Older style F77 routines use set parameter list
- Fortran 5.0 is a combination of F90 routines and new interfaces to older F77 routines
- Takes advantage of F90 language features, such as size() function.  Example, LDA no longer needs to be set, picked up by size (A,1)
- Besides a shorter, cleaner parameter list, user errors are reduced if they have less variables to set

## F90: Simplify Argument Lists

**Simple and Flexible User Interface**
- Short list of required arguments
- Full depth and control via optional arguments
- Maintains full backward compatibility
**Example:**
**Original: ****BCLSJ (FCN, JAC, M, N, XG, IB, XL, XU, XS, FS, IP, RP, X, FV, FJ, LDFJ)**
**New:      ****BCLSJ (FCN, JAC, M, IB, XL, XU, X)**

**Notes:**
New optional arguments allow developer to specify variables or leave them out.  If they leave them out, the code will use a default value.  
Shortens code, making it faster to write, easier to track and debug 
Feature from F90 protocol that has been incorporated in all applicable areas
The example illustrates the simplified argument list when using IMSL subroutine BCLSJ.  

## IMSL Fortran 5.0 Standards Operators and Generic Functions

- Commonly used Linear Algebra functions
- Matrix operations and utilities
- MPI functions and utilities - Used for solving certain problems over a heterogeneous network
- Use the Message Passing Interface
- Box data types are used to broadcast data to different nodes

## IMSL Fortran 5.0 Standards Error Handler

- IMSL has its own error handler – errors are different than compiler or system errors
- ERSET – sets behavior of the error handler for printing and stopping for different errors
- IERCD – retrieves last error code, if any.  These are specific to a routine
- N1RTY – retrieves last error type. Corresponds to the Error Severity Level
- Environment variable used to retrieve IMSL Error message for newer F90 routines; others built in

## IMSL Fortran 5.0 Standards Error Handler (cont)

- Error severity Levels (Default action)
- Level 1  Note (Print:NO, Stop:NO)
- Level 2  Alert (Print:NO, Stop:NO)
- Level 3  Warning (Print:YES, Stop:NO)
- Level 4  Fatal (Print:YES, Stop:YES)
- Level 5  Terminal (Print:YES, Stop:YES)

## BLAS

- Basic Linear Algebra Subroutines
- IMSL supplies our own
- Vendor BLAS usually provide better performance, optimized for compiler and platform
- IMSL BLAS in IMSLBLAS.lib and IMSLBLAS.DLL

## IMSL Fortran 5.0 andHigh Performance Computing

- www.vni.com

## Expanded Support for SMP New for v. 5.0

**Expanded support for SMP/OpenMP multiprocessor environments**
- Expanded number of routines directly enabled for SMP
- Directly enabled computationally intensive algorithms for SMP in the areas of:
- Linear Systems and matrix manipulation
- Eigensystem Analysis
- Fast Fourier Transforms

**Notes:**
Overall shared memory parallel processing strategy for version 5.0 of the IMSL Library.

Note:  Due to limitations on leveraging the NEC MathKeisan BLAS in 64-bit mode, the current version of the IMSL Library leverages linear algebra performance enhancements only.

## Parallel LU Performance

**Notes:**
This slide illustrates the scalability of using the IMSL Fortran Library over multiple CPUs of the SX-6.  Note the scalability as one moves from 1 to 8 CPUs, especially for large problem sizes.

## Support for DMP 

**Fortran 90 functions**
**Nonnegative Least-Squares **
**Bounded Least-Squares  **
**Box Data Type  - (Many problems, same size and type.)**
**MPI Modules**
**ScaLAPACK Utilities**

**Notes:**
Overall shared memory parallel processing strategy for version 5.0 of the IMSL Library.

Note:  Due to limitations on leveraging the NEC MathKeisan BLAS in 64-bit mode, the current version of the IMSL Library leverages linear algebra performance enhancements only.

## Thread Safe Fortran

**OpenMP-based implementation**
- !$OMP  PARALLEL DO
-     DO I=1,NUMBER_INTERVALS
-         CALL UVMIF(F, XGUESS(I), 	+                 BOUND(I), X(I))
-         FX(I)=F(X(I))
-     END DO
- !$OMP  END PARALLEL DO

**Notes:**
Overall shared memory parallel processing strategy for version 5.0 of the IMSL Library.

Note:  Due to limitations on leveraging the NEC MathKeisan BLAS in 64-bit mode, the current version of the IMSL Library leverages linear algebra performance enhancements only.

## Thread Safe Fortran (cont)

- Platform must support OpenMP 2.0
- IBM AIX, XL Fortran 8.1.1.0
- 64-bit
- Sun Solaris, Sun One Studio 8
- 32 and 64-bit
- Multiple copies of existing tests used for OpenMP testing - all platforms.

## Parallel Constrained LSQ

- Dimension M x N; as N increases computational efficiency increases
- User provides partitioning by blocks of columns in array IPART
**IPART(1:2, 1:max(1,MP_NPROCS))**
- MP_NPROCS is the number of processors stored in the communicator

## Parameter list for Nonnegative LSQ

**Call parallel_nonnegative_lsq (A, B, X, & RNORM, W, INDEX, IPART)**
- A is the input matrix of dimension MxN
- B is the right-hand side vector
- X is the solution vector, where X _> _0
- RNORM is the residual vector
- W is the dual vector
- INDEX is the vector that contains constraint information
- IPART array containing the partition information

## Box Data Type

- Linear operators and generic functions designed to compute multiple data sets on multiple CPU systems
- Linear algebra functions (inverse, pseudo-inverse, solvers, etc.)
- SVD, Eigensystem analysis, QR
- FFTs

## MPI Modules

- Modules MPI_setup_int, MPI_node_int
*Following a call to *MP_SETUP()*, *MPI_node_int* will contain:*
- Number of processors
- Rank of a processor
- Communicator information
- Usage priority order of the node systems

## Printing Error Messages

- MP_SETUP(‘Final’)
- Print IMSL error messages
- Messages printed by nodes from largest rank to smallest
- Also terminates MPI execution

## ScaLAPACK Interface Support

- Modules assist in identifying mistakes in usage at compile time
- Argument mismatches
- Missing arguments
- ScaLAPACK_Support includes all module information
- ScaLAPACK_int, PBLAS_int, BLACS_int, Tools_int, LAPACK_Int

## ScaLAPACK Communication Support

- ScaLAPACK_IO_int
- Contains interfaces for ScaLAPACK_Read and ScaLAPACK_Write routines
- ScaLAPACK_Read
- Reads data from a file and transmits data into the 2-d block-cyclic form
- ScaLAPACK_Write
- Writes block-cyclic matrix data to a file
*Synchronize reads and writes for multiple processes*

## IMSL Fortran Chapter Overview

- www.vni.com

## IMSL Chapter Overview

- IMSL Library documentation is divided into three distinct sections but one actual library
- Math
- Math Special Functions
- Stat
- Some overlap, but chapters are separated roughly by functionality

## IMSL Fortran Mathematics 

- Chapter 1: Linear Systems
- Chapter 2: Eigensystem Analysis
- Chapter 3: Interpolation and Approximation
- Chapter 4: Integration and Differentiation
- Chapter 5: Differential Equations
- Chapter 6: Transforms
- Chapter 7: Nonlinear Equations
- Chapter 8: Optimization
- Chapter 9: Basic Matrix/Vector Operations
- Chapter 10: Linear Algebra Operators and Generic Fcns
- Chapter 11:Utilities

## IMSL Fortran Mathematics Special Functions

- Chapter 1: Elementary Functions
- Chapter 2: Trigonometric and Hyperbolic
- Chapter 3: Exponential Integrals and Related
- Chapter 4: Gamma Function and Related
- Chapter 5: Error Function and Related
- Chapter 6: Bessel Functions
- Chapter 7: Kelvin Functions

## IMSL Fortran MathematicsSpecial Functions (cont)

- Chapter 8: Airy Functions
- Chapter 9: Elliptic Integrals
- Chapter 10: Elliptic and Related Functions
- Chapter 11: Probability Distribution Functions and Inverses
- Chapter 12: Mathieu Functions
- Chapter 13: Miscellaneous Functions

## IMSL Fortran Statistics

- Chapter 1: Basic Statistics
- Chapter 2: Regression Chapter 3: Correlation
- Chapter 4: Analysis of Variance
- Chapter 5: Categorical and Discrete Data Analysis
- Chapter 6: Nonparametric Statistics
- Chapter 7: Tests of Goodness-of-Fit and Randomness
- Chapter 8: Time Series Analysis and Forecasting
- Chapter 9: Covariance Structures and Factor Analysis
- Chapter 10: Discriminant Analysis

## IMSL Fortran Statistics (cont)

- Chapter 11: Cluster Analysis
- Chapter 12: Sampling
- Chapter 13: Survival Analysis, Life Testing and Reliability
- Chapter 14: Multidimensional Scaling
- Chapter 15: Density and Hazard Estimation
- Chapter 16: Line Printer Graphics
- Chapter 17: Probability Distribution Functions and Inverses
- Chapter 18: Random Number Generation
- Chapter 19: Utilities
- Chapter 20: Mathematical Support

## Applied Parallel ComputingDr. Richard HansonSenior Scientist, Numerical Methods

- www.vni.com

## Preview

- Rationale for parallel computation
- A constrained optimization example – solving linear inequalities
- Outline of communication needed for the example
- Review of the presentation

## Why Use Parallel Computing?

- The problem is large, so not all data fits in a single memory; or:
- Using multiple processes is expected to improve performance
- The improvements are significant, so _*inevitable complications*_ are acceptable

## Whither Parallel Methods

- Code authors are motivated to use portable packages.  This often implies use of MPI.
- The mathematics and algorithm of the application determine where parallel technology improves performance.
- This is the spirit of our example, illustrating the solution of a single large optimization problem.

## Constrained Optimization Example

- Solve system of linear inequalities
- “Small” length solutions are needed.
- Problem originates from an update:

## Solution is the update and a minimum value of            is desired.
Thus 

- Thus

**Constrained Optimization Example - Continued**

## Solution Method – Example/1

- Based on a dual algorithm:
- Solve the constrained least-squares system

## The IMSL Fortran Library has a code exactly for this problem:     PARALLEL_NONNEGATIVE_LSQ
MPI routines are called in the solution algorithm, as required by the logic.
Any number of processors can be used.  Code ignores MPI calls if the number of processors is == 1 – no parallelism needed.

- MPI routines are called in the solution algorithm, as required by the logic.
- Any number of processors can be used.  Code ignores MPI calls if the number of processors is == 1 – no parallelism needed.

**Solution Method – Example/2**

## Complications arise, but they come with the territory:
Recast dual problem with the partitioning 

- Recast dual problem with the partitioning

**Solution Method – Example/3**

## The partition is defined in the user’s program.  The columns         are located on processor number 
The data for that column must be generated or copied to the processor before the routine is used.
IMSL provides ScaLAPACK_READ for this, but we will not discuss that routine here. 

- The data for that column must be generated or copied to the processor before the routine is used.
- IMSL provides **ScaLAPACK_READ** for this, but we will not discuss that routine here.

**Solution Method – Example/4**

## Routine PARALLEL_NONNEGATIVE_LSQ provides, at each node:
Primal solution 
Dual solution (used later) satisfies   

- Primal solution
- Dual solution (used later) satisfies

**Solution Method – Example/5**

## Obtaining the solution requires each processor compute  
Then compute the sumwith MPI_Reduce( ) routine.

- Then compute the sumwith MPI_Reduce( ) routine.

**Solution Method – Example/6**

## The solution to the inequalities is given by 
   and is first located on a single node.
The solution is then sent to all nodes with the routine MPI_Bcast( ).

-    and is first located on a single node.
- The solution is then sent to all nodes with the routine MPI_Bcast( ).

**Solution Method – Example/7**

## Solution Method – Example/8

- Constraint residuals are given by
- Use MPI_Gather( ) to construct
- Sharpen results with the dual variables:

## Review of Example

- Complete code is given in the IMSL Fortran  Library, Math Chapter 1
- Using parallel computing technologies carries weight of increased complexity in the software development – no avoidance.
- Using IMSL library software, e. g. **PARALLEL_NONNEGATIVE_LSQ,** saves development time.

## Questions

- www.vni.com

## Additional Visual Numerics Product Offerings and Services

- www.vni.com

## IMSLTM Numerical Libraries

- IMSL C Numerical Library (CNL)
- Written in C
- Over 370 threadsafe, math, stat and                    finance algorithms
- JMSLTM Numerical Library for JavaTM Programmers
- 100% Java with IMSL numerics and charting
- IMSL Fortran Numerical Library
- Over 1,000 world-renowned algorithms
- High Performance Computing

**Notes:**

The IMSL Fortran Library 5.0
This new generation of Fortran algorithms is optimized for multiprocessor and other high-performance systems. Fortran Library 5.0 includes the world-renowned IMSL mathematics and statistics algorithms. It also has operators and function modules that provide faster and more natural programming through an object-oriented approach. 

The IMSL C Numerical Library (CNL)
CNL is written in C for C/C++ programmers and is based on the world's most widely used IMSL algorithms, taking full advantage of the language features of C. CNL has over 370 proven, accurate, and thread safe mathematical and statistical algorithms. 

JMSL, IMSL technology for JavaTM Programmers
JMSL is a collection of mathematical, statistical and charting classes, written in 100% JavaTM, that provides the ability to develop network-centric, cost-effective applications. Based on the time-tested, accurate, and reliable IMSL repository of algorithms, JMSL is ideal for JavaTM programmers who need to develop stand-alone or web-enabled applications that require reliable mathematical and statistical functionality, with the added benefit of 2D charting.

## Typical IMSLTM Techniques

- Regression
- Linear systems
- Eigensystem analysis
- Optimization
- Non-parametric statistics
- Time series analysis
- Forecasting (GARCH, ARMA)
- Kalman filtering

- Monte Carlo simulations
- Correlation
- Transforms
- Differential equations
- Multi-dimensional scaling
- Probability distribution
- Co-variance structures
- Factor analysis
- ...and more

**Notes:**
Comprehensive offering of numerical analysis techniques in Mathematics and Statistics.

This represents selected areas and there are more.  Within each area shown here are many specific techniques and algorithms.   

## PV-WAVE Family

**PV-WAVE**
- Language for building desktop visual data analysis applications
- Understand and visualize data
**JWAVE**
- Web based software
- Understand what data means from anywhere, at anytime
**TS-WAVE**
- Time Series Analysis application

**Notes:**
PV-WAVE
PV-WAVE is a family of development solutions that allow you to rapidly import, manipulate, analyze and visualize your data. Unlike other products, PV-WAVE includes a sophisticated set of analysis routines basedupon the IMSL Libraries. 
JWAVE
For access to visual analysis applications from anywhere at anytime, Web-based JWAVE allows you to rapidly analyze, visualize and share critical information. 
TS-WAVE
For more specific applications,TS-WAVE is a highly extensible application that delivers time-series data analysis. With this set of comprehensive solutions, PV-WAVE has earned its place as the choice among experts. 

## Technical Consulting Services

- Custom Algorithm and Application Development
- Ongoing Interaction with Customers to Maintain and Improve Applications
- Diverse Backgrounds and Expertise
- Computer Scientists
- Industry experts
- Visual Data Analysis (VDA) experts
- PhDs in various disciplines including math and stat
- Combination of Training and Consulting Expertise