Big data and its sins
David H. Bailey
http://www.davidhbailey.com
Lawrence Berkeley National Laboratory (retired)Computer Science Department, University of California, Davis

November 13, 2015

1 / 38

Big data science: DANGER AHEAD

Supercomputers operating on big data cangenerate nonsense faster than ever before!

Key concerns:
I Are the results statistically sound?
I Are the results numerically reliable?
I Have the results been validated using independent rigorous tests?
I Are the algorithms, data sources and processing methods well documented?

2 / 38

Reproducibility crises in physics and cosmology
I In 2011, an international team of researchers at the Gran Sasso Laboratory in Italy

announced that neutrinos had exceeded the speed of light, thus directly
challenging Einstein’s relativity. However, after months of careful checking, asubtle flaw was found in the measurement apparatus (a major embarrassment).
I In 2013, CERN researchers confirmed the discovery of the long-sought Higgs

boson. But more recently, scientists have raised questions as to whether theparticle discovered is really the Higgs – it might be some other particle or particles
masquerading as the Higgs; additional research studies are required.
I In March 2014, researchers announced with considerable fanfare that they had

detected the fingerprint of the long-hypothesized inflationary epoch, a tiny
fraction after the big bang. Sadly, within a few weeks critics pointed out that
their experimental results might well be due to dust in the Milky Way, pending
better data.
3 / 38

Reproducibility crises in biomedicine, psychology, economics, finance, autos
I In 2011, Bayer researchers reported that they wereable to reproduce only 17 of 67 pharma studies.
I In 2012, Amgen researchers reported that they wereable to reproduce only 6 of 53 cancer studies.
I In August 2015, the Reproduciblity Project inVirginia reported that they were able to reproduce
only 39 of 100 psychology studies.
I In September 2015, the U.S. Federal Reserve wasable to reproduce only 29 of 67 economics studies.
I In 2014-2015, “backtest overfitting” emerged as a

Reproducibility Project staff
Credit: NY Times

major problem in computational finance.
I In March 2014, West Virginia researchers reported that they were unable toreproduce Volkswagen’s claimed emission figures. This has now exploded into amajor international scandal.
4 / 38Parallel computing, 1991
I Many new parallel systems were offered; users

were excited about potential.
I Each vendor claimed theirs was best, citing one

or two selected applications.
I There were few standard benchmarks or testing

methodologies — mostly Livermore Loops andoriginal Linpack-100.
I Overall, the level of rigor and peer review inthe field was disappointingly low.
I In 1991 DHB published a humorous essay

Twelve Ways to Fool the Masses, poking fun atsome of the abuses (authored by DHB, butwith contributions from the NPB team).
5 / 38

Twelve ways to fool the masses (paraphrased)
1. Quote 32-bit performance results, not 64-bit results, but don’t mention this in paper.
2. Present performance figures for an inner kernel, then represent these figures as theperformance of the entire application.
3. Quietly employ assembly code and other low-level language constructs.
4. Scale up the problem size with the number of processors, but omit any mention of this.
5. Quote performance results projected to a full system.
6. Compare your results against scalar, unoptimized code on conventional systems.
7. When run times are compared, compare with an old code on an obsolete system.
8. Base Mflop/s rates on the operation count of the parallel implementation, instead of thebest practical serial algorithm.
9. Quote performance as processor utilization, parallel speedups or Mflop/s per dollar.
10. Mutilate the algorithm used in the parallel implementation to match the architecture.
11. Measure parallel run times on a dedicated system, but measure conventional run times ina busy environment.
12. If all else fails, show pretty pictures and animated videos, and don’t discuss performance.

6 / 38

NY Times (22 Sep 1991): “Measuring How Fast Computers Really Are”
Excerpts:
I “Rival supercomputer and work station

manufacturers are prone to hype,choosing the performance figures thatmake their own machines look best.”
I “It’s like the Wild West.” [quotingDavid J. Kuck of UIUC].
I “It’s not really to the point ofwidespread fraud, but if people aren’t
a little more circumspect, the entire
field could start to get a bad name.”[quoting DHB].

7 / 38

Examples of abuses: Scaling performance results to full-sized system
In some published papers and conference presentations, runs were performed on
smaller systems, then performance rates were linearly scaled to full-sized systems, insome cases without even clearly disclosing this fact.
Example: 8,192-CPU performance results were linearly scaled to 65,536-CPU results,
simply by multiplying by eight.
Typical excuse: “We can’t afford a full-sized system.”

I D. H. Bailey, “Misleading performance reporting in the supercomputing field,” Scientific

Programming, vol. 1., no. 2 (Winter 1992), 41–151.

8 / 38

Using inefficient algorithms on highly parallel systems
In many cases, inefficient algorithms were employed for highly parallel implementations,requiring many more operations, thus producing artificially high speedup figures andMflop/s rates. Examples:
I Some researchers cited parallel PDE performance based on explicit schemes, forproblems where implicit schemes were known to be significantly more efficient.
I One paper cited performance for computing a 3D discrete Fourier transform bydirect evaluation of the defining formula (8n2 operations), rather than by using afast Fourier transform (5n log2 n).
Both examples violate a basic rule of parallel computing, namely to base parallel
implementations and operation counts (when computing Mflop/s or Gflop/s rates) on
the best practical serial algorithm.
Typical excuse: Other algorithms are “more appropriate” for our parallel system.
9 / 38

Abstract versus details in paper
Abstract of published paper: “The current Connection Machine implementation runs
at 300-800 Mflop/s on a full [64K] CM-2, or at the speed of a single processor of a
Cray-2 on 1/4 of a CM-2.”
I Excerpt from text: “This computation requires 568 iterations (taking 272

seconds) on a 16K Connection Machine.”
In other words, the computation was run on a 16K system, not on a 64K system;
the figures cited in the abstract were merely multiplied by four.
I Excerpt from text: “In contrast, a Convex C210 requires 909 seconds to compute

this example. Experience indicates that for a wide range of problems, a C210 isabout 1/4 the speed of a single processor Cray-2.”
In other words, the computation mentioned in the abstract was actually run on a
Convex system, and a rule-of-thumb scaling factor was used to produce the
Cray-2 rate.
10 / 38

Performance plot: parallel run times (lower) vs vector (upper)
3.5

3Time (Hours)

2.5

2

1.5

1

0.5

0

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

Number of Objects
11 / 38

Data for performance plot
Problem size(x axis)
20
40
80
160
990
9600

Parallel system
run time
8:18
9:11
11:59
15:07
21:32
31:36

Vector system
run time
0:16
0:26
0:57
2:11
19:00
3:11:50*

Details in text of paper:
I In last entry, the 3:11:50 figure is an estimate.
I The vector system code is not optimized.

Note that the parallel system is actually slower than the vector system for all cases,
except for the last (estimated) entry. Also, except for the last entry, all real data in thegraph is in the lower left corner. A log-log plot should have been used instead.
12 / 38

Origins of the NAS Parallel Benchmarks (NPB)I In 1991, a team of 12 researchers at the Numerical Aerodynamic Simulation(NAS) facility (now known as the NASA Advanced Supercomputing facility)
formulated the “NAS Parallel Benchmarks.”
I The original plan was to be a basis for an upcoming supercomputer procurement
for NAS.
I A central goal of the NPB was to test algorithms of importance in computational
aeronautics.
I However, the NPB team recognized from the beginning that the benchmarks
should be designed to have wider usage — hopefully to help clear the hype andconfusion in the field.
The resulting paper (below) has now been awarded the 2015 “Test of Time Award”
from the ACM / IEEE Supercomputing Conference.
I D. H. Bailey, E. Barszcz, J. T. Barton, D. S. Browning, R. L. Carter, L. Dagum, R. A. Fatoohi,

P. O. Frederickson, T. .A. Lasinski, R. S. Schreiber, H. D. Simon, V. Venkatakrishnan and
S. Weeratunga, “The NAS Parallel Benchmarks: Summary and preliminary results,” Proceedingsof Supercomputing 1991, ACM/IEEE Computer Society, 1991, 158–165.

13 / 38

2015: New ways to fool the masses in HPC
I Cite performance rates for a run with only one processor core active in a shared-

memory multi-core node, producing artificially inflated performance (since there is
no shared memory interference) and wasting resources (since most cores are idle).
I

Example: Cite performance on “1024 cores,” even though the code was run on 1024
multicore nodes, one core per node, with 15 out of 16 cores idle on each node.

I Claim that since one is using a graphics processing unit (GPU) system, the mostefficient algorithms must be discarded in favor of “more appropriate” algorithms(recall the experience with parallel algorithms).
I Run the test code many times, but only include the best performance rate in thepaper (recall the experience of recent pharmaceutical trials).
I Employ special hardware, operating system or compiler settings that are notappropriate for real-world production usage (recall the recent Volkswagen scandal).

14 / 38

ICERM report on numerical reproducibility
Numerical round-off error and numerical differences are greatly magnified ascomputational simulations are scaled up to run on highly parallel systems. As a result,
it is increasingly difficult to determine whether a code has been correctly ported to a
new system, because computational results quickly diverge from standard benchmark
cases. And it is doubly difficult for other researchers, using independently written
codes and distinct computer systems, to reproduce published results.

I V. Stodden, D. H. Bailey, J. M. Borwein, R. J. LeVeque, W. Rider and W. Stein, “Setting thedefault to reproducible: Reproducibility in computational and experimental mathematics,” 2012,
http://www.davidhbailey.com/dhbpapers/icerm-report.pdf.

15 / 38

Numerical reproducibility
Many applications routinely use either 32-bit or 64-bit IEEE arithmetic, and employ
fairly simple algorithms, assuming that all is well. But problems can arise:
I Large-scale, highly parallel simulations, running on systems with hundreds ofthousands or millions of processors greatly magnify numerical sensitivities.
I Certain applications with highly ill-conditioned linear systems are particularly
vulnerable.
I Large summations, especially those involving +/- terms and cancellations, canproduce errors.
I Long-time, iterative simulations (such as molecular dynamics or climate models)
are notoriously non-reproducible.
I Computations to resolve small-scale phenomena often require high precision to
produce numerically reproducible results.
I Studies in computational physics or experimental mathematics often require hugeprecision levels.

16 / 38

Analysis of collisions at the Large Hadron Collider
I The 2012 discovery of the Higgs boson at the ATLAS experiment in the LHC

relied crucially on the ability to track charged particles with exquisite precision (10microns over a 10m length) and high reliability (over 99% of roughly 1000
charged particles per collision correctly identified).
I Software: 5 millions line of C++ and Python code, developed by roughly 2000

physicists and engineers over 15 years.
I Recently, in an attempt to speed up the calculation, researchers found that merely

changing the underlying math library resulted in some collisions being missed or
misidentified.
Questions:
I How serious are these numerical difficulties?
I How can they be tracked down?
I How can the library be maintained, producing numerically reliable results?
17 / 38

Approaches to solve accuracy and numerical reproducibility problems
I Employ an expert numerical analyst to re-examine every algorithm employed in thecomputation to ensure that only the most stable known schemes are being used.
I Carefully analyze every step of the computation to ascertain the level of numericalsensitivity at each step.
I Employ interval arithmetic for large portions of the application (which greatly

increases run time and code complexity).
I Employ some “smart” tools to help identify the numerically sensitive parts of an

application, then use higher-precision arithmetic arithmetic if needed to fix them.
In most cases, the last item is the only practical solution.

18 / 38

Numerical analysis expertise among U.C. Berkeley graduates
Of the 2010 U.C. Berkeley graduating class, 870 were in disciplines likely to require
technical computing:
I Division of Mathematical and Physical Sciences (Math, Physics, Statistics).
I College of Chemistry.
I College of Engineering (including Computer Science).

Other fields (not counted) that will likely involve significant computing:
I Biology, geology, medicine, economics, psychology, sociology.

Enrollment in numerical analysis courses:
I Math 128A (Introductory numerical analysis required of math majors): 219.
I Math 128B (A more advanced course, required to do serious work): 24.

Conclusion: Fewer than 2% of Berkeley graduates who will do technical computing
have had rigorous training in numerical analysis.
19 / 38

Enhancing reproducibility with selective usage of high-precision arithmetic
Problem: Find the arc length of the irregular functionPg(x ) = x + 0≤k≤10 2−k sin(2k x ), over the interval (0, π)(using 107 abscissa points).
3.0

I If this computation is done with ordinary double

precision arithmetic, the calculation takes 2.59 seconds
and yields the result 7.073157029008510.
I If it is done using all double-double arithmetic (31-digit

2.5

2.0

1.5

1.0

0.5

accuracy), it takes 47.39 seconds seconds and yields theresult 7.073157029007832 (correct to 15 digits).
I But if only the summation is changed to double-double,

the result is identical to the double-double result (to 15
digits), yet the computation only takes 3.47 seconds.

0.5

1.0

1.5

2.0

2.5

3.0

Graph of g(x ) =P
x + 0≤k≤10 2−k sin(2k x ),over (0, π).

20 / 38

U.C. Berkeley’s CORVETTE project and the “Precimonious” toolObjective: Develop software facilities to find and ameliorate numerical anomalies inlarge-scale computations:
I Test the level of numerical accuracy required for an application.
I Delimit the portions of code that are inaccurate.
I Search the space of possible code modifications.
I Repair numerical difficulties, including usage of high-precision arithmetic.
I Navigate through a hierarchy of precision levels (32-bit, 64-bit, 80-bit or higher as

needed).
The current version of this tool is known as “Precimonious.”

I C. Rubio-Gonzalez, C. Nguyen, H. D. Nguyen, J. Demmel, W. Kahan, K. Sen, D. H. Bailey and

C. Iancu, “Precimonious: Tuning assistant for floating-point precision,” Proceedings of SC13,
Nov 2013.
21 / 38

Precimonious in action for the arc length computation
Original code:

Automatically tuned code:

int main() {int i, n = 1000000;
long double h, t1, t2, dppi;
long double s1;...
for (i = 1; i <= n; i++) {t2 = fun (i * h);s1 = s1 + sqrt (h*h + (t1-t2)*(t1-t2));
t1 = t2;}
return 0;}

int main() {int i, n = 1000000;
double h, t1, t2, dppi;
long double s1;...
for (i = 1; i <= n; i++) {t2 = fun (i * h);s1 = s1 + sqrt (h*h + (t1-t2)*(t1-t2));
t1 = t2;}
return 0;}

The tuned code produces the same answers, to 15-digit accuracy, yet runs faster.

22 / 38

John Gustafson’s new “unum” system for floating-point
Gustafson proposes a radical new definition of floating-point arithmetic, based onunums (short for “universal numbers”), a flexible format that contains a “ubit” todesignate that the true number lies between two adjacent binary floats.
Example: Avogadro’s constant = 6.022 × 1023 is (29 bits total):
fracubit exp size frac size
sign exp
11001101 111111100001 1
111
1011
0
Advantages:
I Obeys commutative law, transitive law and other laws of algebra.
I No rounding error, overflow, underflow, negative zero.
I Consistent treatment of positive and negative infinity and NaN.
I Safe to parallelize; portable between systems.
I Often requires fewer bits than conventional IEEE 64-bit arithmetic.

23 / 38

Sample problem (due to Jean-Michel Muller)
Define:E (z) = (e z − 1)/z,Q(x ) = |x −p

E (0) = 1,x 2 + 1| − 1/(x +px 2 + 1),

H(x ) = E (Q 2 (x )).
Problem: Compute H(x ) for x = 15, 16, 17, 9999.
I IEEE 32-bit: (0, 0, 0, 0).
I IEEE 64-bit: (0, 0, 0, 0).
I IEEE 128-bit: (0, 0, 0, 0).
I Unum (correct): (1, 1, 1, 1).

24 / 38

Another sample problem (due to Siegfried Rump)
DefineF (x , y ) = (333 + 3/4)y 6 + x 2 (11x 2 y 2 − y 6 − 121y 4 − 2) + (5 + 1/2)y 8 + x /(2y )Problem: Compute F (77617, 33096).
I IEEE 32-bit: 1.172603.
I IEEE 64-bit: 1.1726039400531.
I IEEE 128-bit: 1.172603940053178.
I Unum (correct to 23 digits): −0.82739605994682136814116 . . ..

25 / 38

Unum: Current status and plans
I Gustafson’s book has been published by CRC Press (see below).
I A full implementation is nearly complete in C.
I Much testing and analysis will be done.
I Will require a flexible working precision level.
I Hardware implementation?

For additional details:
I John Gustafson, The End of Error: Unum Computing, CRC Press, 2015.

26 / 38

Aren’t 64 bits enough?
There has been considerable resistance in the scientific computing community to thenotion that more than 64-bit arithmetic is not only useful, but may be essential insome scientific computations.
Many believe:
I Physical reality fundamentally does not require high precision, beyond, say, 64-bit

IEEE arithmetic, for any computation.
I Numerical sensitivity problems are always due to the usage of inferior algorithms.
I Conversion costs are always too high, and the added computational costs arealways too great.
I The usage of high-precision arithmetic is “cheating” or “sinful” (because it is too

easy?).

27 / 38

Innocuous example where standard 64-bit precision is inadequate
Problem: Find a polynomial to fit the data (1, 1048579, 16777489,
84941299, 268501249, 655751251, 1360635409, 2523398179, 4311748609) forarguments 0, 1, · · · , 8. The usual approach is to solve the linear system:






Pn n

k=1 xk..
Pn .

n
k=1 xkPn
Pnk=1 xk2
k=1 xk..
P . n+1
k=1 xk

···
···
..
.
···

Pn

  Pnna0k
Pn k=1 xn+1
Pn k=1 yk  a1  
k=1 xkk=1 xk yk 

 
=





..
.
..
  ..  

.
.
Pn
Pn
n
2n
an
k=1 xk yk
k=1 xk

A 64-bit computation (e.g., using Matlab, Linpack or LAPACK) fails to find thecorrect polynomial in this instance, even if one rounds results to nearest integer.
However, if Linpack routines are converted to use double-double arithmetic (31-digit
accuracy), the above computation quickly produces the correct polynomial:f (x ) = 1 + 1048577x 4 + x 8 = 1 + (220 + 1)x 4 + x 8
28 / 38

Innocuous example, cont.
The result on the previous page can be obtained with double precision using Lagrange
interpolation or the Demmel-Koev algorithm. But few scientists, outside of expert
numerical analysts, are aware of these schemes.
Besides, even these schemes fail for higher-degree problems.
For example: (1, 134217731, 8589938753, 97845255883, 549772595201,
2097396156251, 6264239146561, 15804422886323, 35253091827713,
71611233653971, 135217729000001, 240913322581691, 409688091758593)
is generated by:f (x ) = 1 + 134217729x 6 + x 12 = 1 + (227 + 1)x 6 + x 12
Lagrange interpolation and Demmel-Koev fail for this problem, but a straightforward
Linpack scheme, implemented with double-double arithmetic, works fine.
29 / 38

Free software for high-precision computation
1. ARPREC. Arbitrary precision, with numerous algebraic and transcendental
functions. High-level interfaces for C++ and Fortran.
http://crd-legacy.lbl.gov/˜dhbailey/mpdist.
2. GMP. Low-level package for arbitrary precision integer and floating-point
arithmetic. http://gmplib.org.
3. MPFR. C library for arbitrary-precision floating-point computations with exact
rounding, based on GMP. http://www.mpfr.org.
4. MPFR++. High-level C++ interface to MPFR.
http://perso.ens-lyon.fr/nathalie.revol/software.html.
5. MPFUN2015. Similar to ARPREC, but based on MPFR, with a Fortran interface.
http://crd-legacy.lbl.gov/˜dhbailey/mpdist.
6. QD. Performs “double-double” (31 digits) and “quad-double” (62 digits)
arithmetic. High-level interfaces for C++ and Fortran-90.
http://crd-legacy.lbl.gov/˜dhbailey/mpdist.
30 / 38

Some applications where high precision is useful or essential
1. Planetary orbit calculations (32 digits).
2. Supernova simulations (32–64 digits).
3. Climate modeling (32 digits).
4. Coulomb n-body atomic system simulations (32–120 digits).
5. Schrodinger solutions for lithium and helium atoms (32 digits).
6. Electromagnetic scattering theory (32–100 digits).
7. Scattering amplitudes of fundamental particles (32 digits).
8. Discrete dynamical systems (32 digits).
9. Theory of nonlinear oscillators (64 digits).
10. The Taylor algorithm for ODEs (100–600 digits).
11. Ising integrals from mathematical physics (100–1000 digits).
12. Problems in experimental mathematics (100–50,000 digits).
I D. H. Bailey, R. Barrio, and J. M. Borwein, “High precision computation: Mathematical physics

and dynamics,” Applied Mathematics and Computation, vol. 218 (2012), pg. 10106–10121.
31 / 38

Long-term planetary orbit calculations
Researchers have recognized for centuries that planetary orbits exhibit chaotic behavior:“The orbit of any one planet depends on the combined motions of all the planets,
not to mention the actions of all these on each other. To consider simultaneously allthese causes of motion and to define these motions by exact laws allowing of
convenient calculation exceeds, unless I am mistaken, the forces of the entire human
intellect.”[Isaac Newton, Principia, 1687]

Long-term simulations of planetary orbits using double precision do fairly well for long
periods, but then fail at certain key junctures.
Researchers have found that double-double or quad-double arithmetic is occasionallyrequired to avoid severe inaccuracies, even if other techniques are employed to reduce
numerical error.
I G. Lake, T. Quinn and D. C. Richardson, “From Sir Isaac to the Sloan survey: Calculating thestructure and chaos due to gravity in the universe,” Proc. of the 8th ACM-SIAM Symp. on
Discrete Algorithms, 1997, pg. 1–10.
32 / 38

Supernova simulations
I Researchers at LBNL have used quad-double arithmetic

to solve for non-local thermodynamic equilibrium
populations of iron and other atoms in the atmospheresof supernovas.
I Iron may exist in several species, so it is necessary tosolve for all species simultaneously.
I Since the relative population of any state from thedominant state is proportional to the exponential of theionization energy, the dynamic range of these values canbe very large, and cancellations may occur.
I Quad-double arithmetic (62 digits) was employed toreduce these errors.

P. H. Hauschildt andE. Baron, “The numericalsolution of the expanding
stellar atmosphere problem,”
Journal Computational andApplied Mathematics, vol. 109(1999), pg. 41–63.

33 / 38

Climate modeling: High-precision for reproducibility
I Climate and weather simulations are fundamentallychaotic: if microscopic changes are made to the current
state, soon the future state is quite different.
I In practice, computational results are altered even if

minor changes are made to the code or the system.
I This numerical variation is a major nuisance for code

maintenance.
I He and Ding found that by using double-double

arithmetic in two key inner loops, most of thisnumerical variation disappeared.

Y. He and C. Ding, “Using
accurate arithmetics to
improve numericalreproducibility and stability inparallel applications,” Journalof Supercomputing, vol. 18,
no. 3 (Mar 2001), pg.

34 / 38

Coulomb n-body atomic system simulations
I Alexei Frolov of Queen’s University in Canada has used high-precision arithmetic

to solve a generalized eigenvalue problem that arises in Coulomb n-bodyinteractions.
I Matrices are typically 5,000 x 5,000 and are very nearly singular.
I Computations typically involve massive cancellation, and high-precision arithmetic

must be employed to obtain numerically reproducible results.
I Frolov has also computed elements of the Hamiltonian matrix and the overlap

matrix in four- and five-body systems.
I These computations typically require 120-digit arithmetic.

Frolov: “We can consider and solve the bound state few-body problems ... beyond ourimagination even four years ago.”
I D. H. Bailey, R. Barrio, and J. M. Borwein, “High precision computation: Mathematical physics

and dynamics,” Applied Mathematics and Computation, vol. 218 (2012), pg. 10106–10121.
35 / 38

Taylor’s method for ODEs with high-precision arithmetic
1 period - TIDES (16 digits)
16 periods -TIDES (300 digits)

45
40
35

z

30
25
20
15
10
5
0
−10

x

0

10

−20

0

y

20

First point TIDES (16 digits)First-Last point TIDES (300 digits)
Last point TIDES (16 digits)

Numerical integration of the L25-R25 unstable orbit for the Lorenz model during 16
time periods using the TIDES code with 300 digits, and 1 time period using 15 digits.

36 / 38

Discovering new results in mathematics and mathematical physics
Methodology:
1. Compute various mathematical entities (limits, infinite series sums, definite
integrals, etc.) to very high precision, typically 100–100,000 digits.
2. Use algorithms such as PSLQ to recognize these numerical values in terms ofwell-known mathematical constants.
3. When results are found experimentally, seek formal proofs of the discovered
relations.

If mathematics describes an objective world just like physics, there is no reason why
inductive methods should not be applied in mathematics just the same as in physics.
— Kurt Godel
37 / 38

Summary
I Numerical reproducibility is looming as a major challenge for large-scale scientificcomputations.
I One practical solution is to apply intellgient tools that judiciously employ

higher-precision arithmetic for numerically sensitive portions of the code.
I Some interesting computations in experimental mathematics and mathematicalphysics require even more precision — hundreds or even many thousands of digits.
I Such computations are now feasible using modern high-performance computing

systems, together with relatively easy-to-use software facilities.

This talk is available at
http://www.davidhbailey.com/dhbtalks/dhb-bigdata-sins.pdf.
38 / 38