Package: xsum
Fast Exact Summation Using Small and Large Superaccumulators (XSUM)In applications like optimization or finding the sample mean of data, it is
desirable to use higher accuracy than a simple summation. Where in a simple
summation, rounding happens after each addition.
An exact summation is a way to achieve higher accuracy results.XSUMis a library for performing exact summation using
super-accumulators. It provides methods for exactly summing a set of
floating-point numbers, where using a simple summation and the rounding which
happens after each addition could be an important factor in many applications.This library is an easy to use header-only cross-platform C++11 implementation
and also contains Python bindings (please see the example).The main algorithm is taken from the original C libraryFUNCTIONS FOR EXACT SUMMATIONdescribed
in the paper"Fast Exact Summation Using Small and Large Superaccumulators,"byRadford M. Neal.The code is rewritten in C++ and amended with more functionalities with the goal
of ease of use. The provided Python bindings provide theexact summationinterface in a Python code.The C++ code also includes extra summation functionalities, (parallel reduction
on multi-core architectures) which are especially useful in high-performance
message passing libraries (likeOpenMPIandMPICH). Where binding a user-defined global summation
operation to anophandle can subsequently be used inMPI_Reduce,MPI_Allreduce,MPI_Reduce_scatter,andMPI_Scanor a similar calls.NOTE:To see or use or reproduce the results of the original
implementation reported in the paperFast Exact Summation Using Small and Large Superaccumulators, by Radford M.
Neal, please refer toFUNCTIONS FOR EXACT SUMMATION.UsageXSUMlibrary presents two objects, or super-accumulatorsxsum_small_accumulatorandxsum_large_accumulator. It also provides methods
for summing floating-point numbers and rounding to the nearest floating-point
number.A small superaccumulator uses sixty-seven 64-bit chunks, each with 32-bit
overlap with the next one. This accumulator is the preferred method for summing
a moderate number of terms. A large superaccumulator uses 4096 64-bit chunks and
is suitable for big summations. A small superaccumulator is also a component of
the large superaccumulator [1].XSUMlibrary provides two interfaces to use superaccumulators. The first
one is a function interface, which takes input and produces output, and in the
second one, supperaccumulators are represented as classes (xsum_smallandxsum_large.)C++xsum_small_accumulatorandxsum_large_accumulator, both have a default
constructor, thus they do not need to be initialized. Addition operation is
simply axsum_add,// A small superaccumulatorxsum_small_accumulatorsacc;// Adding values to the small accumulator saccxsum_add(&sacc,1.0);xsum_add(&sacc,2.0);// Large superaccumulatorxsum_large_accumulatorlacc;// Adding values to the large accumulator laccxsum_add(&lacc,1.0e-3);xsum_add(&lacc,2.0e-3);orxsum_small, andxsum_largeobjects can simply be used as,// A small superaccumulatorxsum_smallsacc;// Adding values to the small accumulator saccsacc.add(1.0);sacc.add(2.0);// Large superaccumulatorxsum_largelacc;// Adding values to the large accumulator lacclacc.add(1.0e-3);lacc.add(2.0e-3);One can also add a vector of numbers to a superaccumulator.// A small superaccumulatorxsum_small_accumulatorsacc;// Adding a vector of numbersdoublevec[]={1.234e88,-93.3e-23,994.33,1334.3,457.34,-1.234e88,93.3e-23,-994.33,-1334.3,-457.34};xsum_add(&sacc,vec,10);std::vector<double>v={1.234e88,-93.3e-23,994.33,1334.3,457.34,-1.234e88,93.3e-23,-994.33,-1334.3,-457.34};xsum_add(&sacc,v);the same withxsum_small, andxsum_largeobjects as,// A small superaccumulatorxsum_smallsacc;// Adding a vector of numbersdoublevec[]={1.234e88,-93.3e-23,994.33,1334.3,457.34,-1.234e88,93.3e-23,-994.33,-1334.3,-457.34};sacc.add(vec,10);std::vector<double>v={1.234e88,-93.3e-23,994.33,1334.3,457.34,-1.234e88,93.3e-23,-994.33,-1334.3,-457.34};sacc.add(v);The squared norm of a vector (sum of squares of elements) and the dot product of
two vectors (sum of products of corresponding elements) can be added to a
superaccumulator usingxsum_add_sqnormandxsum_add_dotrespectively.For exmaple,// A small superaccumulatorxsum_small_accumulatorsacc;// Adding a vector of numbersdoublevec1[]={1.e-2,2.,3.};doublevec2[]={1.e-3,2.,3.};// Add dot product of vectors to a small superaccumulatorxsum_add_dot(&sacc,vec1,vec2,3);withxsum_small, andxsum_largeobjects as,// A small superaccumulatorxsum_smallsacc;// Adding a vector of numbersdoublevec1[]={1.e-2,2.,3.};doublevec2[]={1.e-3,2.,3.};// Add dot product of vectors to a small superaccumulatorsacc.add_dot(vec1,vec2,3);When it is needed, one can simply use thexsum_initto reinitilize the
superaccumulator.xsum_small_accumulatorsacc;xsum_add(&sacc,1.0e-2);...// Reinitilize the small accumulatorxsum_init(&sacc);...or withxsum_smallobject as,xsum_smallsacc;sacc.add(1.0e-2);...// Reinitilize the small accumulatorsacc.init();...The superaccumulator can be rounded as,xsum_small_accumulatorsacc;xsum_add(&sacc,1.0e-15);....doubles=xsum_round(&sacc);where,xsum_roundis used to round the superaccumulator to the nearest
floating-point number.Withxsum_small, andxsum_largeobjects we do as,xsum_smallsacc;sacc.add(1.0e-15);....doubles=sacc.round();where,roundis used to round the superaccumulator to the nearest
floating-point number.Twosmallsuperaccumulators can be added together.xsum_addcan be used to
add the second superaccumulator to the first one without doing any rounding. Twolargesuperaccumulator can also be added in the same way. In the case of the
two large superaccumulators, the second one internally will be rounded to a
small superaccumulator and then the addition is done.// Small superaccumulatorsxsum_small_accumulatorsacc1;xsum_small_accumulatorsacc2;xsum_add(&sacc1,1.0);xsum_add(&sacc2,2.0);xsum_add(&sacc1,&sacc2);// Large superaccumulatorsxsum_large_accumulatorlacc1;xsum_large_accumulatorlacc2;xsum_add(&lacc1,1.0);xsum_add(&lacc2,2.0);xsum_add(&lacc1,&lacc2);or as,// Small superaccumulatorsxsum_smallsacc1;xsum_smallsacc2;sacc1.add(1.0);sacc2.add(2.0);// Add a small accumulator sacc2 to the first accumulator sacc1sacc1.add(sacc2);// Large superaccumulatorsxsum_largelacc1;xsum_largelacc2;lacc1.add(1.0);lacc2.add(2.0);// Add a large accumulator lacc2 to the first accumulator lacc1lacc1.add(lacc2);Asmallsuperaccumulator can also be added to alargeone,// Small superaccumulatorxsum_small_accumulatorsacc;xsum_add(&sacc,1.0e-10);...// Large superaccumulatorxsum_large_accumulatorlacc;xsum_add(&lacc,2.0e-3);...// Addition of a small superaccumulator to a large onexsum_add(&lacc,&sacc);Withxsum_small, andxsum_largeobjects we do as,// Small superaccumulatorxsum_smallsacc;sacc.add(1.0e-10);...// Large superaccumulatorxsum_largelacc;lacc.add(2.0e-3);...// Addition of a small superaccumulator to a large onelacc.add(sacc);The large superaccumulator can be rounded to a small one as,xsum_large_accumulatorlacc;xsum_small_accumulatorsacc=xsum_round_to_small(&lacc);or as,xsum_largelacc;xsum_small_accumulatorsacc=lacc.round_to_small();ExampleTwo simple examples on how to use the library:#include<iomanip>#include<iostream>#include"xsum/xsum.hpp"usingnamespacexsum;intmain(){// Large superaccumulatorxsum_large_accumulatorlacc;doubleconsta=0.7209e-5;doubles=0;for(inti=0;i<10000;++i){xsum_add(&lacc,a);s+=a;}std::cout<<std::setprecision(20)<<xsum_round(&lacc)<<"\n"<<std::setprecision(20)<<s<<"\n";}or axsum_smallorxsum_largeobjects can simply be used as,#include<iomanip>#include<iostream>#include"xsum/xsum.hpp"usingnamespacexsum;intmain(){// Large superaccumulatorxsum_largelacc;doubleconsta=0.7209e-5;doubles=0;for(inti=0;i<10000;++i){lacc.add(a);s+=a;}std::cout<<std::setprecision(20)<<lacc.round()<<"\n"<<std::setprecision(20)<<s<<"\n";}One can compile the code as,g++simple.cpp-std=c++11-O3-osimpleoricpcsimple.cpp-std=c++11-O3-fp-model=double-osimplerunning thesimplewould result,./simple0.0720900000000000013010.072089999999998641278MPI reduction example (MPI_Allreduce)To use a superaccumulator in high-performance message
passing libraries, first we need an MPI datatype of a superaccumulator, and then
we define a user-defined global operationXSUMthat can subsequently be used
inMPI_Reduce,MPI_Aallreduce,MPI_Reduce_scatter, andMPI_Scan.The below example is a simple demonstration of the use of a superaccumulator on
multiple processors, where the final summation across all processors is desired.#include<mpi.h>#include<iomanip>#include<iostream>#include"xsum/myxsum.hpp"#include"xsum/xsum.hpp"usingnamespacexsum;intmain(){// Initialize the MPI environmentMPI_Init(NULL,NULL);// Get the number of processesintworld_size;MPI_Comm_size(MPI_COMM_WORLD,&world_size);// Get the rank of the processintworld_rank;MPI_Comm_rank(MPI_COMM_WORLD,&world_rank);// Create the MPI data type of the superaccumulatorMPI_Datatypeacc_mpi=create_mpi_type<xsum_large_accumulator>();// Create the XSUM user-opMPI_OpXSUM=create_XSUM<xsum_large_accumulator>();doubleconsta=0.239e-3;doubles(0);xsum_large_accumulatorlacc;for(inti=0;i<1000;++i){s+=a;xsum_add(&lacc,a);}MPI_Allreduce(MPI_IN_PLACE,&s,1,MPI_DOUBLE,MPI_SUM,MPI_COMM_WORLD);MPI_Allreduce(MPI_IN_PLACE,&lacc,1,acc_mpi,XSUM,MPI_COMM_WORLD);if(world_rank==0){std::cout<<"Rank =  "<<world_rank<<", sum   =  "<<std::setprecision(20)<<a*1000*world_size<<", sum 1 =  "<<std::setprecision(20)<<s<<", sum 2 =  "<<std::setprecision(20)<<xsum_round(&lacc)<<std::endl;}// Free the created user-opdestroy_XSUM(XSUM);// Free the created MPI data typedestroy_mpi_type(acc_mpi);// Finalize the MPI environment.MPI_Finalize();}mpic++mpi_simple.cpp-std=c++11-O3-osimplerunning the above code using 4 processors would result,mpirun-np4./simpleRank=0,sum=0.95600000000000007194,sum1=0.95599999999998419575,sum2=0.95600000000000007194PythonThe provided Python bindings provide theexact summationinterface in a
Python code.Python requirementsYou need Python 3.6 or later to runxsum. You can have multiple Python
versions (2.x and 3.x) installed on the same system without problems.To install Python 3 for different Linux flavors, macOS and Windows, packages
are available athttps://www.python.org/getit/Using pippipis the most popular tool for installing Python packages, and the one
included with modern versions of Python.xsumcan be installed withpip:pipinstallxsumNote:Depending on your Python installation, you may need to usepip3instead ofpip.pip3installxsumDepending on your configuration, you may have to runpiplike this:python3-mpipinstallxsumUsing pip (GIT Support)pipcurrently supports cloning overgitpipinstallgit+https://github.com/yafshar/xsum.gitFor more information and examples, see thepip installreference.Using condacondais the package management tool for Anaconda Python installations.Installingxsumfrom theconda-forgechannel can be achieved by
addingconda-forgeto your channels with:condaconfig--addchannelsconda-forgeOnce theconda-forgechannel has been enabled,xsumcan be
installed with:condainstallxsumIt is possible to list all of the versions ofxsumavailable on your platform
with:condasearchxsum--channelconda-forgePython examplesfromxsumimport*importnumpyasnp# A small superaccumulatorsacc=xsum_small_accumulator()# Adding values to the small accumulator saccxsum_add(sacc,1.0)xsum_add(sacc,2.0)# Large superaccumulatorlacc=xsum_large_accumulator()# Adding values to the large accumulator laccxsum_add(lacc,1.0e-3)xsum_add(lacc,2.0e-3)One can also add a vector of numbers to a superaccumulator.fromxsumimport*importnumpyasnp# A small superaccumulatorsacc=xsum_small_accumulator()a=np.arange(0,1,0.1)# Adding a vector of numbersxsum_add(sacc,a)print("sum       ={:.20f}".format(np.sum(a)))print("Exact sum ={:.20f}".format(xsum_round(sacc)))or axsum_smallorxsum_largeobjects can simply be used as,fromxsumimport*importnumpyasnp# A small superaccumulatorsacc=xsum_small()a=np.arange(0,1,0.1)# Adding a vector of numberssacc.add(a)print("sum       ={:.20f}".format(np.sum(a)))print("Exact sum ={:.20f}".format(sacc.round()))running thesimple.pyscript would result,python./simple.pysum=4.50000000000000088818
Exactsum=4.50000000000000000000ReferencesNeal, Radford M., "Fast exact summation using small and large superaccumulators,"arXiv e-prints, (2015)https://www.cs.toronto.edu/~radford/xsum.software.htmlhttps://gitlab.com/radfordneal/xsumContributingCopyright (c) 2020, Regents of the University of Minnesota.All rights reserved.Contributors:Yaser AfsharLicenseLGPLv2.1