Kahan Summation Algorithm

23 Dec, 2021
Prerequisites: Rounding off errors, Introduction to the floating-point representationKahan summation algorithm, also known as compensated summation and summation with the carry algorithm, is used to minimize the loss of significance in the total result obtained by adding a sequence of finite-precision floating-point numbers. This is done by keeping a separate running compensation (a variable to accumulate small errors). 
Reason for Loss of Significance: 
As we know in Java language we have two primitive floating-point types, float and double, with single-precision 32-bit and double-precision 64-bit format values and operations specified by IEEE 754. That is, they are represented in a form like:
SIGN FRACTION * 2EXP
For example, 0.15625 = (0.00101)2, which in floating-point format is represented as: 1.01 * 2-3. However, not all fractions can be represented exactly as a fraction of a power of two. For example, 0.1 = (0.000110011001100110011001100110011001100110011001100110011001... )2 and thus cannot be stored inside a floating-point variable.
Therefore, floating-point error/loss of significance refers to when a number that cannot be stored as it is in the IEEE floating-point representation and repetitively some arithmetic operation is performed on it. This leads to some unexpected value and the difference between the expected and the obtained value is the error.
Below is an implementation that simulates the significance error:
C++
Java
Python3
Javascript
// C++ program to illustrate the// floating-point error#include<bits/stdc++.h>using namespace std; double floatError(double no){    double sum = 0.0;    for(int i = 0; i < 10; i++)    {        sum = sum + no;    }    return sum;} // Driver codeint main(){    cout << setprecision(16);    cout << floatError(0.1);} // This code is contributed by rutvik_56
// Java program to illustrate the// floating-point error public class GFG {     public static double floatError(double no)    {        double sum = 0.0;        for (int i = 0; i < 10; i++) {            sum = sum + no;        }        return sum;    }     public static void main(String[] args)    {         System.out.println(floatError(0.1));    }}
# Python3 program to illustrate the# floating-point error def floatError(no):    sum = 0.0    for i in range(10):        sum = sum + no    return sum if __name__ == '__main__':    print(floatError(0.1)) # This code is contributed by mohit kumar 29
<script>// Javascript program to illustrate the// floating-point error function floatError(no){    let sum = 0.0;        for (let i = 0; i < 10; i++) {            sum = sum + no;        }        return sum;} document.write(floatError(0.1));   // This code is contributed by patel2127</script>
0.9999999999999999
 
Note: The expected value for the above implementation is 1.0 but the value returned is 0.9999999999999999. Therefore, in this article, a method to reduce this error by using Kahan’s Summation algorithm is discussed. 
Kahan Summation Algorithm: The idea of the Kahan summation algorithm is to compensate for the floating-point error by keeping a separate variable to store the running time errors as the arithmetic operations are being performed. This can be visualised by the following pseudocode: 
function KahanSum(input)
    var sum = 0.0                    
    var c = 0.0                      
    for i = 1 to input.length do     
                                     
        var y = input[i] - c         
        var t = sum + y              
        c = (t - sum) - y            
                                     
        sum = t                      
                                     
    next i                          

    return sum
In the above pseudocode, algebraically, the variable c in which the error is stored is always 0. However, when there is a loss of significance, it stores the error in it. 
Below is the implementation of the above approach:
C++
Java
Javascript
Python3
// C++ program to illustrate the// Kahan summation algorithm #include <bits/stdc++.h>using namespace std; // Function to implement the Kahan// summation algorithmdouble kahanSum(vector<double> &fa){    double sum = 0.0;     // Variable to store the error    double c = 0.0;     // Loop to iterate over the array    for(double f : fa)    {        double y = f - c;        double t = sum + y;                 // Algebraically, c is always 0        // when t is replaced by its        // value from the above expression.        // But, when there is a loss,        // the higher-order y is cancelled        // out by subtracting y from c and        // all that remains is the        // lower-order error in c        c = (t - sum) - y;        sum = t;    }    return sum;} // Function to implement the sum// of an arraydouble sum(vector<double> &fa){    double sum = 0.0;     // Loop to find the sum of the array    for(double f : fa)    {        sum = sum + f;    }    return sum;} // Driver codeint main(){    vector<double> no(10);    for(int i = 0; i < 10; i++)    {        no[i] = 0.1;    }      // Comparing the results      cout << setprecision(16);    cout << "Normal sum: " << sum(no) << " \n";    cout << "Kahan sum: " << kahanSum(no);    } // This code is contributed by ajaykr00kj
// Java program to illustrate the// Kahan summation algorithm public class GFG {     // Function to implement the Kahan    // summation algorithm    private static double kahanSum(double... fa)    {        double sum = 0.0;         // Variable to store the error        double c = 0.0;         // Loop to iterate over the array        for (double f : fa) {             double y = f - c;            double t = sum + y;             // Algebraically, c is always 0            // when t is replaced by its            // value from the above expression.            // But, when there is a loss,            // the higher-order y is cancelled            // out by subtracting y from c and            // all that remains is the            // lower-order error in c            c = (t - sum) - y;             sum = t;        }         return sum;    }     // Function to implement the sum    // of an array    private static double sum(double... fa)    {        double sum = 0.0;         // Loop to find the sum of the array        for (double f : fa) {            sum = sum + f;        }         return sum;    }     // Driver code    public static void main(String[] args)    {        double[] no = new double[10];        for (int i = 0; i < no.length; i++) {            no[i] = 0.1;        }         // Comparing the results        System.out.println("Normal sum: " + sum(no));        System.out.println("Kahan sum: " + kahanSum(no));    }}
<script> // Javascript program to illustrate the// Kahan summation algorithm // Function to implement the Kahan// summation algorithmfunction kahanSum(fa){    let sum = 0.0;      // Variable to store the error    let c = 0.0;     // Loop to iterate over the array    for(let f = 0; f < fa.length; f++)    {        let y = fa[f] - c;        let t = sum + y;         // Algebraically, c is always 0        // when t is replaced by its        // value from the above expression.        // But, when there is a loss,        // the higher-order y is cancelled        // out by subtracting y from c and        // all that remains is the        // lower-order error in c        c = (t - sum) - y;         sum = t;    }    return sum;} // Function to implement the sum// of an arrayfunction sum(fa){    let sum = 0.0;      // Loop to find the sum of the array    for(let f = 0; f < fa.length; f++)    {        sum = sum + fa[f];    }    return sum;} // Driver codelet no = new Array(10);for(let i = 0; i < no.length; i++){    no[i] = 0.1;} // Comparing the resultsdocument.write("Normal sum: " +               sum(no) + "<br>");document.write("Kahan sum: " +               kahanSum(no).toFixed(1) + "<br>"); // This code is contributed by unknown2108 </script>
# Python3 program to illustrate the# Kahan summation algorithm # Function to implement the Kahan# summation algorithmdef kahanSum(fa):    sum = 0.0     # Variable to store the error    c = 0.0     # Loop to iterate over the array    for f in fa:        y = f - c        t = sum + y         # Algebraically, c is always 0        # when t is replaced by its        # value from the above expression.        # But, when there is a loss,        # the higher-order y is cancelled        # out by subtracting y from c and        # all that remains is the        # lower-order error in c        c = (t - sum) - y        sum = t     return sum  # Driver codeif __name__ == "__main__":    no = [0.0] * 10    for i in range(10):        no[i] = 0.1     # Comparing the results    print("Normal sum: ", sum(no))    print("Kahan sum: ", kahanSum(no))
Normal sum: 0.9999999999999999
Kahan sum: 1.0
 
Time Complexity: O(N), where N is the length of the list.Auxiliary Space: O(1)
mohit kumar 29
rutvik_56
ajaykr00kj
patel2127
unknown2108
pankajsharmagfg
ashutoshsinghgeeksforgeeks
amartyaghoshgfg
Algorithms
Write From Home
Algorithms
Writing code in comment?
Please use ide.geeksforgeeks.org,
generate link and share the link here.