Reproducibility and variable precision computing
David H. Bailey
Lawrence Berkeley Natlional Laboratory (retired)
University of California, Davis, Department of Computer Science
This talk is available at:https://www.davidhbailey.com/dhbtalks/dhb-sc-2019.pdf

1 / 17

Numerical reproducibility in scientific computing
A December 2012 workshop on reproducibility in scientific computing, held at Brown
University, USA, noted that
Science is built upon the foundations of theory and experiment validated andimproved through open, transparent communication. ...
Numerical round-off error and numerical differences are greatly magnified ascomputational simulations are scaled up to run on highly parallel systems. As
a result, it is increasingly difficult to determine whether a code has beencorrectly ported to a new system, because computational results quickly
diverge from standard benchmark cases. And it is doubly difficult for otherresearchers, using independently written codes and distinct computer
systems, to reproduce published results.
I V. Stodden, D. H. Bailey, J. M. Borwein, R. J. LeVeque, W. Rider and W. Stein, “Setting the default toreproducible: Reproducibility in computational and experimental mathematics,”
http://www.davidhbailey.com/dhbpapers/icerm-report.pdf.
2 / 17

Reproducibility problems in a Large Hadron Collider code
I The 2012 discovery of the Higgs boson at the ATLAS experiment in the LHC

relied crucially on the ability to track charged particles with exquisite precision (10microns over a 10m length) and high reliability (over 99% of roughly 1000
charged particles per collision correctly identified).
I Software: five million line of C++ and Python code, developed by roughly 2000
physicists and engineers over 15 years.
Recently, in an attempt to speed up the calculation, researchers found that merely
changing the underlying math library resulted in some collisions being missed or
misidentified.
Questions:
I How extensive are these numerical difficulties?
I How can numerically sensitive code be tracked down?
I How can the library be maintained, producing numerically reliable andreproducible results?
3 / 17

How to enhance numerical reproducibility and solve accuracy problems
1. Employ an expert numerical analyst to examine every algorithm employed in thecode, to ensure that only the most stable and efficient schemes are being used.
2. Employ an expert numerical analyst to analyze every section of code for numericalsensitivity.
3. Employ interval arithmetic for large portions of the application (which will greatly
increases run time and code complexity).
4. Employ variable precision arithmetic, assisted with software tools to help
determine where extra precision is needed and where it is not.
#4 is the only practical solution for real-world computing.

4 / 17

Advantages to variable precision computing
1. Faster processing in sections of code that can use lower precision.
2. Better cache utilization (because more data fit in cache).
3. Lower offline data storage costs.
4. Lower energy costs.
But effective usage of variable precision requires:
1. Appropriate software tools.
2. A more sophisticated mathematical framework,
3. Some fundamental rethinking of what reproducibility means in a variable precisioncontext.

5 / 17

Commonly used formats for floating-point computing
Formalname
IEEE 16-bit(none)(none)IEEE 32-bit
IEEE 64-bit
IEEE 80-bit
IEEE 128-bit(none)(none)(none)

Nickname“IEEE half”“ARM half”“bfloat16”“IEEE single”“IEEE double”“IEEE extended”“IEEE quad”“double double”“quad double”“multiple”

Sign
1
1
1
1
1
1
1
1
1
1

Number of bits
Exponent Mantissa
5
10
5
10
8
7
7
24
11
52
15
64
15
112
11
104
11
208
variesvaries

Hidden
1
1
1
1
1
0
1
2
4
variesDigits
3
3
2
7
15
19
34
31
62
varies6 / 17

Variable precision applications
Low precision (16-bit or less):
I Graphics.
I Artificial intelligence.
I Machine learning.
Mixed precision (16-32-64-bit):
I New mixed precision algorithms for common linear algebra operations.
I New iterative refinement schemes: Perform initial iterations in 16-bit or 32-bit
precision, then only the last one or two using full 64-bit precision.
I Using new floating-point compression techniques for on-the-fly storage and massstorage.
I M. Baboulin, A. Buttari, J. Dongarra, J. Kurzak, J. Langou, J. Langou, P. Luszczek and S. Tomov,“Accelerating scientific computations with mixed precision algorithms”, Comp. Phys. Comm., vol. 180(2009), 2526–2533, http://dx.doi.org/10.1016/j.cpc.2008.11.005.
I A. Buttari, J. Dongarra, J. Kurzak, P. Luszczek and S. Tomov, “Using mixed precision for sparse matrix
computations to enhance the performance while achieving 64-bit accuracy,” ACM TOMS, vol. 34 (2008),
art. 17, http://doi.acm.org/10.1145/1377596.1377597.
I P. Lindstrom, “Fixed-rate compressed floating-point arrays,” IEEE Trans. Vis. Comp. Graphics, vol. 20(2014), 2674–2683, Dec 2014, https://doi.org/10.1109/TVCG.2014.2346458.

7 / 17

DeepMind’s AlphaGo Zero teaches itself to play Go
I In October 2017, DeepMind researchers programmed an AI

computer with the rules of Go, then had the program playitself — literally teaching itself with no human input.
I After just three days of training, the resulting program“AlphaGo Zero” defeated their earlier program (which had
defeated Ke Jie, the highest rated human) 100 games to 0.
I The Elo rating of Ke Jie, the world’s highest rated Go

player, is 3661. After 40 days of training, AlphaGo Zero’s
Elo rating was over 5000.
I AlphaGo Zero is as far ahead of the world champion as theworld champion is ahead of a good amateur.
I “The latest AI can work things out without being taught,” The Economist, 21 Oct 2017,https://www.economist.com/news/science-and-technology/
21730391-learning-play-go-only-start-latest-ai-can-work-things-out-without.
8 / 17

Quad precision in computational biology
I Stanford researchers are using the Constraint-Based Reconstruction and Analysis(COBRA) approach, a linear programming based scheme, to study biologicalprocesses such as metabolism and macromolecular synthesis.
I Researchers found that 32-bit arithmetic is “not useful,” and that 64-bit
arithmetic “may not ensure adequate accuracy,” but selected usage of IEEE
128-bit (“quad”) produces solutions of “exceptional accuracy.”
The authors conclude: We believe that quadruple-precision solutions are now practicalfor multiscale [linear programming] applications ..., and that they justify increased
confidence as systems biologists build ever-larger models.
I Ding Ma and Michael A. Saunders, “Solving multiscale linear programs using the simplex method in
quadruple precision,” in M. Al-Baali et al. (eds.), Numerical Analysis and Optimization, Springer
Proceedings in Mathematics and Statistics, vol. 134, DOI 10.1007/978-3-319-17689-5 9.
I D. Ma, L. Yang, R. M. T. Fleming, I. Thiele, B. O. Palsson and M. A. Saunders, “Reliable and efficientsolution of genome-scale models of metabolism and macromolecular expression,” Nature Scientific
Reports, vol. 7, 40863 (2017), https://www.nature.com/articles/srep40863.
9 / 17Other extra-high precision applications
1. Planetary orbit calculations (32 digits).
2. Supernova simulations (32–64 digits).
3. Climate modeling (16-32 digits).
4. Scattering amplitudes of fundamental particles (32 digits).
5. Discrete dynamical systems (32 digits).
6. Coulomb n-body atomic system simulations (32–120 digits).
7. Using the Taylor algorithm for ODEs (100–300 digits).
8. Recognizing integrals from the Ising theory of mathematical physics in terms of
simple mathematical identities (200-1000 digits).
9. Finding minimal polynomials connected to the Poisson equation of mathematicalphysics (1000–64,000 digits).
I D. H. Bailey, R. Barrio, and J. M. Borwein, “High precision computation: Mathematical physics anddynamics,” Applied Mathematics and Computation, vol. 218 (2012), pg. 10106–10121,
https://www.davidhbailey.com/dhbpapers/hpmpd.pdf.
I D. H. Bailey, J. M. Borwein, J. Kimberley and W. Ladd, ”Computer discovery and analysis of large
Poisson polynomials,” Experimental Mathematics, 27 Aug 2016, vol. 26, pg. 349-363,
https://www.davidhbailey.com/dhbpapers/poisson-res.pdf.

10 / 17

The Poisson potential function
In 2012, Richard Crandall, while investigating techniques to sharpen images, notedthat each pixel was given by a form of the 2-D Poisson potential function:
1 X cos(mπx ) cos(nπy )φ2 (x , y ) = 2
π m,n odd
m2 + n2
In a 2013 study, we numerically discovered, and then proved the intriguing fact that for
rational (x , y ),1
φ2 (x , y ) = log απwhere α is algebraic, i.e., the root of a some integer polynomial of degree m.
By computing high-precision numerical values of φ2 (x , y ) for various specific rational x
and y , and applying a variation of the PSLQ integer relation algorithm, we were able
to produce the explicit minimal polynomials for α in numerous specific cases.
I D. H. Bailey, J. M. Borwein, R. E. Crandall and J. Zucker, “Lattice sums arising from the Poisson
equation,” Journal of Physics A: Mathematical and Theoretical, vol. 46 (2013), 115201.
11 / 17Samples of minimal polynomials found by multipair PSLQ
s
5
6
7
8
9

10

Minimal polynomial corresponding to x = y = 1/s:
1 + 52α − 26α2 − 12α3 + α4
1 − 28α + 6α2 − 28α3 + α4
−1 − 196α + 1302α2 − 14756α3 + 15673α4 + 42168α5 − 111916α6 + 82264α7−35231α8 + 19852α9 − 2954α10 − 308α11 + 7α12
1 − 88α + 92α2 − 872α3 + 1990α4 − 872α5 + 92α6 − 88α7 + α8
−1 − 534α + 10923α2 − 342864α3 + 2304684α4 − 7820712α5 + 13729068α6−22321584α7 + 39775986α8 − 44431044α9 + 19899882α10 + 3546576α11−8458020α12 + 4009176α13 − 273348α14 + 121392α15−11385α16 − 342α17 + 3α18
1 − 216α + 860α2 − 744α3 + 454α4 − 744α5 + 860α6 − 216α7 + α8These computations are very expensive and require exceedingly high numeric precision.
The case x = y = 1/32, for instance, required 10,000-digit arithmetic and ran for 45
hours. Other runs have required up to 64,000-digit arithmetic.

12 / 17

Kimberley’s formula for the degree of the polynomial
Based on our preliminary results, Jason Kimberley of the University of Newcastle,
Australia observed that the degree m(s) of the minimal polynomial associated with thecase x = y = 1/s appears to be given by the following:
Set m(2) = 1/2. Otherwise for primes p congruent to 1 mod 4, set m(p) = int2 (p/2),
where int denotes greatest integer, and for primes p congruent to 3 mod 4, setm(p) = int (p/2)(int (p/2) + 1). Then for any other positive integer s whose prime
factorization is s = p1e1 p2e2 · · · prer ,?

m(s) = 4r −1

rY2(ei −1)pim(pi ).

i=1

After additional very high-precision computations and some rather sophisticatedmathematical work, this formula has now been proven.
I D. H. Bailey, J. M. Borwein, J. Kimberley and W. Ladd, “Computer discovery and analysis of large
Poisson polynomials,” Experimental Mathematics, 27 Aug 2016, vol. 26, pg. 349–363,
https://www.davidhbailey.com/dhbpapers/poisson-res.pdf.
13 / 17

192-degree minimal polynomial found by multipair PSLQ for x = y = 1/35
−1−127936α1−119729568α2+86259615424α3−63414593420752α4−23761356230294720α5−21521006149475920864α6+611939502140287079872α7−368241444434585386649704α8+34534137997535622208800832α9+190487328667825240861480928α10+70731866698900460686569086656α11−30639321248717091789168538879600α12+1922209242084456061370070376570688α13−92225937246234731111918315818791008α14+5237085972458845348795423777776516544α15−235187168740813424983476512954481139732α16+7429223399159195923652315325576416851776α17−183888805409443551177519325744878574722592α18+3271185405201792963719366162166966763587520α19−30299858585212592953207274837044089721625936α20−359992131877582010695076712447360628122541504α21+21102025339314733846086441916632636325491195040α22−476414747523697562008419279360098519444810134848α23+7169682338305684610208768903330082567075749930088α24−77147038334570163492295981291240215533152479746240α25+576164594448741453064385715717563185909190291963744α26−2736845608824484767533372475869861068967680432337984α27+14259726273826198867906144815986191950918103600120720α28−418898418536517746458185180042147268305073686192184768α29+11576727630588055309375415098495616370526328495842993184α30−215864800528660839585002162871715414654518484882221347136α31+2990075428515164844805618172618994337159659201108555443326α32−32103405822691698883721869657536188329165809065032597536320α33+266563559238616545234045835785848715794979563412588549450912α34−1588318769762074127932309766905276050093368042694667151097024α35+4203983024865006644081516487096952711041894782888827508128720α36+45741795572596990194281792252600295928845217432176402329342656α37−888295322846459251495779433538199608179522746791003115766467360α38+9318868074863097497435367536227145321575550333899160360356887616α39−75360568030730663716613169159158032198635749984420852664531432312α40+511904470165868318936871843389229757000581029769096726546021748160α41−3023546278817700158319248501909046229060364677626955289951128897760α42+15762439699463291788277556458704672972787957041647009977980115608384α43−72736254087237112123433235647299134668422519990232924303557659578512α44+294655977143150502457165877012542972911987116693855121083332478262976α45−1023418666617067140855492177628022043641503497647927192890895515496096α46+2872077837131433597858520147550093715024860106120898508958083768560704α47−5244656757384067630568130031166724732476128944651080774800327815250116α48−4375176099824059357126327245955265067024929918868045662842672259365696α49+106846263928061290720995820179791755195031418846948198250527939644422688α50−723639781157123885224054475278025279616831219096501715148796353565656000α51+3859388734169523941501510320228124865152568559427283813598867448036672464α52−18252524309522593867333335673661626604394606878814374964432136569382704704α53+77801314121940709085266334072671847312436136451041737155601244216316832608α54−295924487894698517430363594146438746974018698578472622597771256623091267264α55+990883094386701201740101120896676716687418747972363663196338999871139087736α56−2890647897108739627601089028825831836951153432467718962671967596342224331584α57+7329280206578701233923536747523603867159738577214393279268225554697992024224α58−16529926761499923756105727653427646107756611125773189972311741253023019544512α59+36621401257350141587688985844214102107297318891842289523223570545646432107248α60−97806177157646206090474078202543089426709440278475346399840243186928760712768α61+342578593578669077931375617617876157438332517966938868387799492392630391951328α62−1318494126999138496197980784859753020673610723076044612133782854663456341738176α63+4770648039063802379141989053619421884321933507524681278243277292208295535890193α64−15472388710953109453763060955215809455238864792962040591046439233238251454944640α65+44993126133327412718394996675533962755303579158603905782013384675151509701621696α66−118993678023270686331414029749780909390537125214711043931901528442137124714048640α67+291344068959030996949609862561213601556661749401236700666696596438800158224020448α68−672232544304160020140857460151229781783982323755372371953704546852887742475693952α69+1483756546102154371991621621590988762258290163435879224071206411473630010421819712α70−3160698702072288781771809020419553277725945343183442108817290538424291765729105536α71+6501409535318122589047227869235783252522832570490016457500770280864513874652673648α72−12819483008841743167047686498254296089188806357669061135254105501043843161878338944α73+23940036962388573943673383046497610566473660443396196928270109685668366770350448320α74−41783013070841149166516454269070382975961508352312427230333483156459522180289158272α75+67368007706334464267463678880196649459723255934263643619439168617608299192825564576α76−99642265418236291841000783588241107943576928164973329187609122090423042369846900608α77+135684188062925983721829145899247186881926375911313411751554577363782314702542167104α78−175027359690635875118927274192015444148378662148930449526142186378361018115616958080α79+230610470370976022179942433042240414747465002830049591030281645554186668837721744600α80−345422259636540072058441736406800364089516876786410123401855696720824391150388817792α81+607137279243399399687877325853919130478767608151821751113771985567328949196202362560α82−1146510481188778693690480353627945407015450534575302104964676472156118460868902533760α83+2105965918280150700239337266375899616661414836881685735099986426462071874731653407456α84−3578099460146102835319749769961500032509484008579871514018505538256220639466406781312α85+5536176932749319486998056607166537426841616046655786532661950492192459385736417266240α86−7794809387121374412027723614750591530039194538680900530684292991362557543590881311872α87+10030481053829943456941341746197226060710869001059273241750964485149858086288306297744α88−11855047237622313521872161223741536919023583101964392808122345150250641280790373298048α89+12893688641739677210122894074867960717441055755861579189125047363100403706385492504000α90−12809768930934167317548440121172865343713268055679115994897043173034752329114006909568α91+11262021213965304198804174698825443038955997004185306033241807189003803427064173678496α92−7849339137067651884501051001175657872891640580392350339448446525146908046759472284032α93+2136636160531235808486132100072698791511127607341553178572108601522645635713639494976α94+6182245041445709382582906443600895624739514751663138744452121338925118726793099119488α95−17027139906672640329386998091617550180769534392263392331607751724460799383582736580380α96+29712689442226074490778272413199981836245382131442863784077108073804247980419140795264α97−42913200159533980876837680464079550150161925164315385234076771249584133995790605789888α98+54875995472202404064433213065393198278520963443598737304851154803060885419026758577792α99−63825924492815000947804006660643589720936270079124038361274887833762135234355101020256α100+68414285135348430004939705729313406303651902455163077326777079330134700391931405532544α101−68052822555902228583831595467842429584162408591782046566558459654662872517242957132352α102+63030283713395534150979634221684031444907507016820847420753457009176282665669129552000α103−54395011700122510436311965134146069935113985222440557617340304628640177217303223510128α104+43661984233468332856484141573706612495184170864395287152161473759679906830742322508672α105−32445083300862179709263952078603686506241016652233926920050485448240544236200991135168α106+22121488216739201475462682897890849680076071306863200873172222890471168791244013948544α107−13610378673217079648813094175609528224090224497348403954492135388377641815307093302560α108+7303046170572758067914716975092412590455319569015290620012702916095079596345712424320α109−3131236977889234295288739605819294961076022655796155501587485430182196310350741412160α110+722365801671568238376537172560503547318051932349906390028021898906118213151314346112α111+423601672347880485375916854724736111845460547572098224965700862087464288437477713112α112−789372219071689131389195080081110795264640291084303894471284509727586944366644459136α113+754844732770124186279971574591381270567329047796107073522275693379041299799000185920α114−570721609484121135843526600643872181106278042023747642137438316501500109706499901312α115+373431689502164436293456395379326584419834315995264527851770397586202342215482658208α116−218828293376534306771048774311447660676991404850097946000689151980961553792808627328α117+116819901747981791438375631794807731777629646639313589196533853783698354327272619712α118−57453490833130202085276802289813871676642923245392648322689802787108386720310373760α119+26322973910217009898036214368526156227892184032219090112763726491148743638265345648α120−11422293344043226356444271083191575214531430352803712260273887252165192023955482240α121+4829637193522288826261580288388799608086659826584129673254995173846268064699275584α122−2078817645839949613653484866883184944970522163048142353896259321595643918566571904α123+954520484158500566039633963780071095170152665054125308901179320476061925020693472α124−476059705123351801546660809193814948416900226947841461260532426685030594421013632α125+251019218316831093023612611636151648630754206670792757985712506316495845522315200α126−133387943415348693213086926089912666295139757189579549272926455197093747072689536α127+68669026491757170599503679381653507376133436777604611890688641155842031359807249α128−33465058649746756993954154200626776375881202657693549363818927924447316311308992α129+15265533880702809827068316634690758620560027313985769068567445431127044039098336α130−6486555236218719687502565717159307954250235769707821904032108839967986971592256α131+2562655019815708913497740003785614466295686123950263933555489655843860264778480α132−940718343979230237299701844566444332546715633645270453853660861734418393188288α133+320771113247818349988983708385653420090152866910698288499686276305286454845600α134−101564452363828199943698157849015312966252040558153110340955250960209509456704α135+29840961316238615781884908158117270537546621262752881740317094715665305372024α136−8126376541879784121657187449503319934776774553449947624311536879983067197120α137+2046994405582919610833914133280166599368341899820908062960382811295175875424α138−475319952147035704834648908024038830280982173827054926876842801020099391040α139+101156139413765995787243855087479738205336119604403640417144416030882137040α140−19536944183496787665897200701914355830359619924729440672204518600726485952α141+3365653573540320310956912862417397989082549665698949749718751351520029216α142−500297582644104844305032452251740515847130656744730715102464280225674048α143+59315729762438733681847579101255419553245420619721207125629069846855484α144−4088968402271371424704252106469344820514085294011337888699549616483264α145−410467411100906673223801525015092827907382852275953565568989408591520α146+244373061239573525170364306606773136690590544856331578133540062357184α147−67137173574208581929099251980225280832815704879993996789961828208784α148+15109453726979262114404719422671376896934991181254480301068783496000α149−3041110741874930638456938010940447684323532404585513230577319188704α150+543474006539496836413675511283771054220572953580018501137140774336α151−82818210860123209370721479679382853454718857015790579069608342392α152+10208909053669116781213384172905303479740054958245095883752727616α153−950607878263673381203873433978784840581837837196127719161731872α154+61537081222427663468738382440653874375219870006734261147376320α155−3614460685557548015719619196028506524834155796764598110372912α156+735146337727640447184879802116259476500819273333226229041984α157−195982957610545601975537049907586449965158403571666750972768α158+35410924134603994553881822516846688340985279903764142254528α159−4586047122379261183418802995335871699189992574849634423682α160+461731528371721725878159689755046846967144056271095432896α161−39548982563052670609179355793571460666409257926095752160α162+3249771246267792941295436011847663684436436914874939968α163−278663520870823657308884684973631236730880421051628656α164+23298555433192575099637973614016576299882162116142016α165−1572480211244215449798675303409379604821388457521312α166+64631732188205970561818215170229051834055602691904α167−359811432336185820581657539275402497742419221400α168−81408350985197555231086181640295965142859955520α169−2670526194482509900914512598125562096574018400α170+824346510881081434287176385644420962607200832α171−40421524575366405147127382177082176049208656α172+859850260877732200189610333732782641323968α173−21746942237477507610278314875487367745056α174+1166204976072113679141136387334878097216α175−16255593447525276321168587153278001172α176−1031047179999272558707644762054830656α177+57315125130135149120405695909830560α178−2190911023714361858443237127579840α179+42156348386945234230569584900496α180−400797347138034992145950828864α181−3239264180104926121034061856α182+57518923520963556655145024α183−302326293499892000458856α184−648961315190292320832α185+4804670128512328736α186+97238301302930240α187−106616047418832α188−478181605696α189+208298592α190+25664α191−α192This distinctive semi-elliptical pattern is strong evidence that the result is correct.

14 / 17

The U.C. Berkeley/U.C. Davis “Precimonious” toolObjective: Develop software facilities to find and ameliorate numerical anomalies inlarge-scale computations:
I Facilities to test the level of numerical accuracy required for an application.
I Facilities to delimit the portions of code that are inaccurate.
I Facilities to search the space of possible code modifications.
I Facilities to repair numerical difficulties, including usage of high-precision
arithmetic.
I Facilities to navigate through a hierarchy of precision levels (32-bit, 64-bit or
higher as needed).
The current version of this tool is known as “Precimonious.”
I C. Rubio-Gonzalez, C. Nguyen, H. D. Nguyen, J. Demmel, W. Kahan, K. Sen, D. H. Bailey and C. Iancu,“Precimonious: Tuning assistant for floating-point precision,” Proceedings of SC13, November 2013,https://www.davidhbailey.com/dhbpapers/precimonious.pdf.
I C. Nguyen, C. Rubio-Gonzalez, B. Mehne, K. Sen, C. Iancu, J. Demmel, W. Kahan, W. Lavrijsen,
D. H. Bailey and D. Hough, “Floating-point precision tuning using blame analysis,” Proceedings of the
38th International Conference on Software Engineering (ICSE 2016), 14–22 May 2016,15 / 17
https://www.davidhbailey.com/dhbpapers/blame-analysis.pdf.

Bit-for-bit floating-point replicability between systems
I Although the IEEE floating-point standards guarantee bit-for-bit results (provided

the same rounding mode is used) for individual operations, different systems may
produce different higher-level results, because of non-associativity:(A ⊕ B) ⊕ C 6= A ⊕ (B ⊕ C ).
I Recently some researchers have proposed an addition to the IEEE-754 standard
that would permit guaranteed bit-for-bit replicability, not only on a single systembut also on a parallel system. See references below.
I ARM (a U.K. computer processsor vendor) has implemented a high-precision
accumulator, using an adaptable vector unit.
Dangers: If this standard becomes the default on computer systems:
I Will it lock into place computations that are seriously inaccurate?
I Will it make it more difficult for a user to even be aware that he/she has aproblem with numerical sensitivity and/or inaccuracy?
I P. Ahrens, J. Demmel and H. D Nguyen, “Algorithms for efficient reproducible floating point summation,”
manuscript available from authors.
I J. Riedy and J. Demmel, “Augmented arithmetic operations proposed for IEEE-754 2018,”
http://www.ecs.umass.edu/arith-2018/pdf/arith25_34.pdf.
16 / 17

Research questions
1. How can we certify that 16-bit results, say in machine learning, are numericallymeaningful? Can machine learning community benchmarks help?
2. How can reproducibility be measured and verified in more general types of variable
precision applications?
3. Can techniques for bit-for-bit reproducibility be incorporated in a variable precisionenvironment? What can be done to mitigate the dangers of such facilities?
4. What new opportunities are there for using iterative refinement with variable
precision?
5. What new numerical analysis techniques should be considered in a variable
precision environment (algorithms, error bounds, error estimates, etc.).
6. What software tools are available to automatically or semi-automatically analyze
an application code, identify numerically sensitive spots and propose or implement
remedies? What other tools are needed?
This talk is available at:https://www.davidhbailey.com/dhbtalks/dhb-sc-2019.pdf
17 / 17