Talk:Kahan summation algorithm

message: The algorithm as described is, in fact, Kahan summation as it is described in W. Kahan, Further Remarks On Reducing Truncation Errors, Communications of the ACM, 8(1),40, however, this algorithm only works for either values of y[i] of similar magnitude or in general for increasing y[i] or y[i] << s.
Higham's paperN. J. Higham, The Accuracy Of Floating Point Summation, SIAM Journal of Scientific Computing, 14(4), 783-799 on the subject has a much more detailed analysis, including different summation techniques. I will try to add some of the main statements from that paper to this article if I find the time and there are no objections.
Pedro.Gonnet 16:51, 5 October 2006 (UTC)
thread_title: Some Comments

message: I have seen a variation of this method, as follows.  I wonder if it is well known and how it compares. McKay (talk) 07:24, 2 April 2009 (UTC)
replies: text: Hummm. Well, here is the article's version (as at the time of typing)
 function kahanSum(input, n)
  var sum = input[1]
  var c = 0.0          //A running compensation for lost low-order bits.
  for i = 2 to n
   y = input[i] - c    //So far, so good: c is zero.
   t = sum + y         //Alas, sum is big, y small, so low-order digits of y are lost.
   c = (t - sum) - y   //(t - sum) recovers the high-order part of y; subtracting y recovers -(low part of y)
   sum = t             //Algebraically, c should always be zero. Beware eagerly optimising compilers!
  next i               //Next time around, the lost low part will be added to y in a fresh attempt.
 return sum
Leaving aside the special-feature initialisation of sum, at first sight I'd suggest that the variant you describe might have trouble with the accumulated error steadily increasing as would happen with truncation-based arithmetic, and with rounding the errors would be of varying sign, but the magnitude of their sum would increase, that is, the possible magnitude of c would spread proportional to Sqrt(N). Whereas with the article's version the magnitude of the running deviation is kept small since whenever it becomes larger, its larger part is assimilated into sum. A proper assessment of these matters would require some careful analysis and explanation of the results, that some might decry as looking like Original Research. NickyMcLean (talk) 21:08, 2 April 2009 (UTC)

text: Yes, McKay's version has roughly O(sqrt(n)) error growth, not the O(1) of Kahan. — Steven G. Johnson (talk) 23:30, 11 March 2010 (UTC)
replies: text:  (Note that I corrected the last line of my version from "sum+c" to "sum-c".) You may be right, but in lots of simulations using 109 random numbers, both all of the same sign and of mixed signs, I didn't see an example where these two algorithms gave significantly different answers. Usually the answers were exactly the same and about 1000 times better than naive summation. I think the n1/2 growth of c only becomes significant when n is about ε-2, which is well beyond the practical range. McKay (talk) 05:38, 20 February 2017 (UTC)
thread_title:  A variation 

message: It would be nice if the article included some information on progress on compensated summation since Kahan's original algorithm.  For example, this paper reviews a number of algorithms that improve upon Kahan's accuracy in various ways, e.g. obtaining an error proportional to the square of the machine precision or an error independent of the condition number of the sum (not just the length), albeit at greater computational expense:
:Rump et al, "Accurate floating-point summation part I: faithful rounding", SIAM J. Sci. Comput. 31 (1), p. 189-224 (2008).
All that is mentioned right now is Shewchuk's work, and I'm not sure the description of his work is accurate; from the description in his paper, he's really doing arbitrary precision arithmetic, where the required precision (and hence the runtime and storage) are cleverly adapted as needed for a given computation (changed).
— Steven G. Johnson (talk) 23:46, 11 March 2010 (UTC)
thread_title:  Progress since Kahan 

message: At the end of the Example Working section it says "few systems supply quadruple precision".  I think that the opposite is now true: most systems supply quadruple presision (i.e. 128 bit floating point).  Does anyone disagree?  McKay (talk) 06:25, 11 February 2011 (UTC)
replies: text: Very few systems support it in hardware.  A number of compilers for C and Fortran (but not all by any means) support it in software, and it is absent from many other languages; it depends on how you define "most". — Steven G. Johnson (talk) 02:34, 12 February 2011 (UTC)
thread_title:  quadruple precision 

message: It's claimed in the article that python's fsum uses a method by Shewchuk to attain exact rounding. However, based on Shewchuk's paper it would seem that the sum is not always rounded exactly. Does anyone have an opinion how to better describe fsum's accurasy? Ossi (talk) 14:43, 15 March 2012 (UTC)
replies: text: Shewchuk's paper describes both "exact addition and multiplication" algorithms and "adaptive precision arithmetic" that satisfies any desired error bound, hence my understanding is it can be used to compute results more precise than double which can therefore be exactly rounded.  The Python fsum documentation says that it achieves exactly rounded results as long at as the CPU is IEEE round-to-even double precision (and makes at most a one bit error for CPU's running in extended-precision mode).  Looking at the source code seems to confirm that they do indeed guarantee that the routine "correctly rounds the final result" (given IEEE arithmetic, and not including overflow situations) and provides some more information about the algorithms.  In particular, it seems to use a version of what Shewchuk's paper calls the "FAST-EXPANSION-SUM" algorithm. — Steven G. Johnson (talk) 17:53, 15 March 2012 (UTC)
replies: text: Thank you for the link. It seems to me that based on the source code that it's just "GROW-EXPANSION" rather than "FAST-EXPANSION-SUM" which is used, but that is not important here. I didn't doubt that Shewchuk's algorithms could produce an exactly correct expansion. The reason I thought the sums wouldn't always be exactly rounded is that Shewchuk only discusses methods to approximate the expansions with a single floating point number in a way which aren't always exactly rounded (e.g. using COMPRESS). Python's source seems to use a method which is not from Shewchuk's paper. This was the source of my confusion. Ossi (talk) 00:07, 24 March 2012 (UTC)
thread_title:  Python's fsum 

message: See: http://cage.ugent.be/~klein/papers/floating-point.pdf  --Amro (talk) 18:56, 30 June 2013 (UTC)
thread_title:  Kahan-Babuška variation 

message: The article currently has a sentence:
"This worst-case error is rarely observed in practice, however, because it only occurs if the rounding errors are all in the same direction."
I'm not sure whether it's worth changing, let alone removing, the sentence, but it is remarkably easy to accidentally end up with all rounding errors in the same direction when doing naive summation, especially with single-precision floating-point numbers.
For example, adding the value 1 to an initially zero accumulator 1,000,000,000,000 times will result in a value of 16,777,216, which is 2^24, because as soon as it reaches 2^24, (2^24)+1 rounds back down to 2^24, so it stays there forever.  This sort of catastrophic roundoff error happens pretty much whenever summing 2^24 or more values of the same sign in single-precision.  I've hit it in several different real-world applications before, and it's a case where either increased precision or Kahan summation is absolutely necessary. Ndickson (talk) 23:04, 25 July 2015 (UTC)
replies: text: For what it's worth, my practice is to have the summation decide on a working average early on, say the first supplied number (or the average of some early numbers), and then work with summing (x(i) - w), in the hope (pious) that the resulting summation would not wander far from zero and thereby provoke the misfortune you mention. If seriously worried over the quality of the estimate of w, the approach would be to perform the summation in two passes. The idea here is that many positive numbers are being summed, all say around 12,345, so that otherwise the total just keeps on increasing - until it doesn't. The problem arises much sooner with single-precision numbers of course. NickyMcLean (talk) 11:16, 7 November 2015 (UTC)
thread_title:  Rounding errors all in the same direction 

message: neumaier list misses a next i
I have found improvement in real*8 with g77 only up to 10^19, not up to 10^100
pietro151.29.209.13 (talk) 10:36, 28 September 2017 (UTC)
thread_title:  comment