Parallel Functionality and AutomaticThread Control in PV-WAVE® 10.0
PV-WAVE Performance Improvement Brochure

Rogue Wave Software
April 2012

Rogue Wave Software
5500 Flatiron Parkway,
Suite 200
Boulder, CO 80301, USA
www.roguewave.com

Parallel Functionality and AutomaticThread Control in PV-WAVE® 10.0
PV-WAVE Performance Improvements Brochure
by Rogue Wave Software© 2012 by Rogue Wave Software. All Rights Reserved
Printed in the United States of America

Publishing History:
April 2012

Trademark Information
The Rogue Wave Software name and logo, SourcePro, Stingray, HostAccess, TotalView, IMSL and PV-WAVE areregistered trademarks of Rogue Wave Software, Inc. or its subsidiaries in the US and other countries. HydraExpress,
ThreadSpotter, ReplayEngine, MemoryScape, JMSL, JWAVE, TS-WAVE, and PyIMSL are trademarks of Rogue Wave
Software, Inc. or its subsidiaries. All other company, product or brand names are the property of their respective owners.
IMPORTANT NOTICE: The information contained in this document is subject to change without notice. Rogue Wave
Software, Inc. makes no warranty of any kind with regards to this material, including, but not limited to, the implied
warranties of merchantability and fitness for a particular purpose. Rogue Wave Software, Inc. shall not be liable for errors
contained herein or for incidental, consequential, or other indirect damages in connection with the furnishing,performance, or use of this material.

TABLE OF CONTENTS
Abstract ........................................................................................................................................... 4
Automatic Thread Control (ATC) .................................................................................................... 4
Figure 1 – Copying an Array of BYTEs .......................................................................................... 5
Figure 2 – Addition Using BYTE Values ........................................................................................ 5Figure 3 – Hyperbolic Tangent of Double Complex Values ......................................................... 6Figure 4 – Composition of Several Array Operations .................................................................. 7Figure 5 – Sequence of Array Subscripting Operations ............................................................... 8Figure 6 – Looping Over Array Columns with Addition ............................................................... 9
Newly Parallelized Array Operations ............................................................................................ 10Conclusion ..................................................................................................................................... 10About Rogue Wave Software ........................................................................................................ 10Abstract
During the last several releases of PV-WAVE, there has been a concerted effort to enhanceperformance. For PV-WAVE versions from 8.0 to 9.0 we parallelized the majority ofparallelizable array operations. In PV-WAVE 9.5 we introduced Automatic Thread Control (ATC),an automatically-tuned runtime system that optimizes parallel performance. And for PV-WAVE
10.0, we enhanced ATC, and we parallelized most of the array operations that were still serial,
most notably the multi-dimensional operations like subscripting, tiling, and concatenation.
This document describes the advantages of ATC over manual threading control. It also
highlights the newly parallelized functionality for this most recent release. For a full listing ofOpenMP-enabled functionality refer to the PV-WAVE documentation.

Automatic Thread Control (ATC)ATC is now available for PV-WAVE on Linux platforms, and on Windows platforms where it wasintroduced for PV-WAVE in version 9.5, it has been much enhanced.
PV-WAVE uses OpenMP to parallelize nearly all parallelizable array operations, and ATC is an
automatically-tuned runtime system which automatically adjusts the number of threads so that
it is optimal for any given operation, regardless of array size, data-type, and computingplatform.
ATC circumvents a common problem in array-programming: inefficient thread usage. The
problem is best illustrated with a few plots.
Figures 1-3 are speedup plots for copying a byte array, for adding a scalar to each element of abyte array, and for computing the hyperbolic-tangent of each element in a DCOMPLEX array.
The x-axis indicates array sizes of 2x elements, the y-axis indicates speedups, the green curve
indicates the default method of thread control where a loop is parallel if and only if it exceeds
999 iterations, and the white curve indicates ATC where a loop is run in parallel if and only if it
has been predetermined to be faster than running it serially. The plots were generated on aDell running RHEL6-64 with 2 X5650's each with 6-cores, 12MB cache, and hyper-threading off.

4

www.roguewave.com

Figure 1 – Copying an Array of BYTEs

Figure 2 – Addition Using BYTE Values

5

www.roguewave.com

Figure 3 – Hyperbolic Tangent of Double Complex Values
Figures 1 and 3 show extremes for parallelization threshold (the operation size above which
parallelization is efficient) and Figure 2 is an intermediate case. On this platform as on mostplatforms, parallelization threshold can vary by at least four orders of magnitude from one
operation to the next, so using a fixed threshold impedes the performance of medium-sized
operations with too many threads (Figures 1 and 2) or with not enough threads (Figure 3).
Although medium-sized array operations are computationally cheaper than the larger ones,
they are just as important to optimize because they can sometimes dominate the work in an
unparallelizable loop, itself composed of parallel array operations.
Any attempt to programmatically control the threshold in a large array-based program would
be prohibitively time-consuming for the programmer.
Fortunately, the PV-WAVE programmer can launch a simple script that tunes PV-WAVE to any
Linux or Windows platform. Once tuned to a particular platform, PV-WAVE automaticallyadjusts the number of threads at runtime so that it is always optimal for any operation in any
PV-WAVE application running on that platform.
The previously discussed Figures 1-3 show how ATC improves the performance of individual
element-wise array operations. Figures 4-6 show how it improves the performance of multidimensional array operations and composites of array operations where thepenalties associated with a fixed parallelization threshold can becomecompounded.

6

www.roguewave.com

Figure 4 – Composition of Several Array Operations
Figure 4 is a speedup plot for a composite of the several array operations (SQRT and those indisteuclcart.pro) used to compute the matrix of pair-wise distances between two sets of points.

7

www.roguewave.com

Figure 5 – Sequence of Array Subscripting Operations
Figure 5 shows the performance for a composite of operations used to replace part of a column
in a 2d array with part of a column from another 2d array.

8www.roguewave.com

Figure 6 – Looping Over Array Columns with Addition
Figure 6 shows the performance of a sequence of operations similar to those shown in Figure 5,
but where each replacement column has a scalar added to it.
The six speedup plots are typical of speedup plots for any operation or composite ofoperations, where ATC is seen to improve performance by degrees which vary with theoperation and where the improvements are always limited to the intermediate sizes of theoperation. Regardless of dimensionality, virtually all PV-WAVE array operations are subject to
ATC: initializations, copies, reverses, tilings, type conversions, subscripting, searches,intersections, concatenations, transcendental functions, and logical, relational, arithmetic,
matrix, and tensor operations.

9www.roguewave.com

Newly Parallelized Array Operations
Many previously serial array operations are now parallel and subject to ATC:N-D Subscripting
N-D Index Conversions (INDEX_CONV)
N-D Concatenation (the [] operators)
N-D Tiling (REBIN with Sample keyword)
N-D Cartesian Products (CPROD)
Order Reversals (REVERSE)
Index Generation (LINSPACE)
Type Conversions (BYTE, FIX, INT32, LONG, FLOAT, DOUBLE, COMPLEX, DCOMPLEX)
Complex Utilities (IMAGINARY, CONJ)
Searches (WHERE)
Greatest, Smallest, and Nearest Integer functions (GREAT_INT, SMALL_INT, NINT)
Parallelization of these array operations produces speedups in computational array-based PVWAVE code. For example, the RBF function-fitter RBFIMSCL is a large PV-WAVE application
which makes extensive use of the new parallel array functionality, and for medium to large
problems on multi-core desktop hardware, it now runs more than twice as fast as it did in PVWAVE version 9.50.

Conclusion
For computationally-intensive array-based processing of medium to large data sets, the newly
parallel functionality and Automatic Thread Control in PV-WAVE 10.0 can reduce runtimes bymore than a factor of two over PV-WAVE 9.50.

About Rogue Wave Software
Rogue Wave Software, Inc. is the largest independent provider of cross-platform softwaredevelopment tools and embedded components for the next generation of HPC applications.
Rogue Wave marries High Performance Computing with High Productivity Computing to enable
developers to harness the power of parallel applications and multicore computing. Rogue Wave
products reduce the complexity of prototyping, developing, debugging, and optimizing multiprocessor and data-intensive applications. Rogue Wave customers are industry leaders in the
Global 2000, ISVs, OEMs, government laboratories and research institutions that leveragecomputationally-complex and data-intensive applications to enable innovation and outperform
competitors. For more information, visit http://www.roguewave.com.

10

www.roguewave.com