barty/D4J-Repair huggingface dataset:
File: src/main/java/org/apache/commons/math3/optimization/general/AbstractLeastSquaresOptimizer.java
Issue: new multivariate vector optimizers cannot be used with large number of weights
When using the Weigth class to pass a large number of weights to multivariate vector optimizers, an nxn full matrix is created (and copied) when a n elements vector is used. This exhausts memory when n is large.
This happens for example when using curve fitters (even simple curve fitters like polynomial ones for low degree) with large number of points. I encountered this with curve fitting on 41200 points, which created a matrix with 1.7 billion elements.

Buggy code:
private RealMatrix squareRoot(RealMatrix m) {
        final EigenDecomposition dec = new EigenDecomposition(m);
        return dec.getSquareRoot();
}

Fixed code:
private RealMatrix squareRoot(RealMatrix m) {
    if (m instanceof DiagonalMatrix) {
        final int dim = m.getRowDimension();
        final RealMatrix sqrtM = new DiagonalMatrix(dim);
        for (int i = 0; i < dim; i++) {
           sqrtM.setEntry(i, i, FastMath.sqrt(m.getEntry(i, i)));
        }
        return sqrtM;
    } else {
        final EigenDecomposition dec = new EigenDecomposition(m);
        return dec.getSquareRoot();
    }
}
File: src/main/java/org/apache/commons/math3/dfp/Dfp.java
Issue: Dfp Dfp.multiply(int x) does not comply with the general contract FieldElement.multiply(int n)
In class org.apache.commons.math3.Dfp,  the method multiply(int n) is limited to 0 <= n <= 9999. This is not consistent with the general contract of FieldElement.multiply(int n), where there should be no limitation on the values of n.

Buggy code:
public Dfp multiply(final int x) {
        return multiplyFast(x);
}

Fixed code:
public Dfp multiply(final int x) {
    if (x >= 0 && x < RADIX) {
        return multiplyFast(x);
    } else {
        return multiply(newInstance(x));
    }
}
File: src/main/java/org/apache/commons/math3/distribution/HypergeometricDistribution.java
Issue: HypergeometricDistribution.sample suffers from integer overflow
Hi, I have an application which broke when ported from commons math 2.2 to 3.2. It looks like the HypergeometricDistribution.sample() method doesn't work as well as it used to with large integer values – the example code below should return a sample between 0 and 50, but usually returns -50.

import org.apache.commons.math3.distribution.HypergeometricDistribution;

public class Foo {
  public static void main(String[] args) {
    HypergeometricDistribution a = new HypergeometricDistribution(
        43130568, 42976365, 50);
    System.out.printf("%d %d%n", a.getSupportLowerBound(), a.getSupportUpperBound()); // Prints "0 50"
    System.out.printf("%d%n",a.sample());                                             // Prints "-50"
  }
}

In the debugger, I traced it as far as an integer overflow in HypergeometricDistribution.getNumericalMean() – instead of doing

return (double) (getSampleSize() * getNumberOfSuccesses()) / (double) getPopulationSize();

it could do:

return getSampleSize() * ((double) getNumberOfSuccesses() / (double) getPopulationSize());

This seemed to fix it, based on a quick test.

Buggy code:
public double getNumericalMean() {
    return (double) (getSampleSize() * getNumberOfSuccesses()) / (double) getPopulationSize();
}

Fixed code:
public double getNumericalMean() {
    return getSampleSize() * (getNumberOfSuccesses() / (double) getPopulationSize());
}
File: src/main/java/org/apache/commons/math3/optimization/direct/CMAESOptimizer.java
Issue: CMAESOptimizer does not enforce bounds
The CMAESOptimizer can exceed the bounds passed to optimize.  Looking at the generationLoop in doOptimize(), it does a bounds check by calling isFeasible() but if checkFeasableCount is zero (the default) then isFeasible() is never even called.  Also, even with non-zero checkFeasableCount it may give up before finding an in-bounds offspring and go forward with an out-of-bounds offspring.  This is against svn revision 1387637.  I can provide an example program where the optimizer ends up with a fit outside the prescribed bounds if that would help.

Buggy code:
public double[] repairAndDecode(final double[] x) {
    return
        decode(x);
}

Fixed code:
public double[] repairAndDecode(final double[] x) {
    return boundaries != null && isRepairMode ?
        decode(repair(x)) :
        decode(x);
}
File: src/main/java/org/apache/commons/math3/fraction/Fraction.java
Issue: Fraction percentageValue rare overflow
The percentageValue() method of the Fraction class works by first multiplying the Fraction by 100, then converting the Fraction to a double. This causes overflows when the numerator is greater than Integer.MAX_VALUE/100, even when the value of the fraction is far below this value.
The patch changes the method to first convert to a double value, and then multiply this value by 100 - the result should be the same, but with less overflows. An addition to the test for the method that covers this bug is also included.

Buggy code:
public double percentageValue() {
    return multiply(100).doubleValue();
}

Fixed code:
public double percentageValue() {
    return 100 * doubleValue();
}
File: src/main/java/org/apache/commons/math3/stat/inference/MannWhitneyUTest.java
Issue: Mann-Whitney U Test Suffers From Integer Overflow With Large Data Sets
When performing a Mann-Whitney U Test on large data sets (the attached test uses two 1500 element sets), intermediate integer values used in calculateAsymptoticPValue can overflow, leading to invalid results, such as p-values of NaN, or incorrect calculations.
Attached is a patch, including a test, and a fix, which modifies the affected code to use doubles

Buggy code:
private double calculateAsymptoticPValue(final double Umin,
                                         final int n1,
                                         final int n2)
    throws ConvergenceException, MaxCountExceededException {

    final int n1n2prod = n1 * n2;

    // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
    final double EU = n1n2prod / 2.0;
    final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;

    final double z = (Umin - EU) / FastMath.sqrt(VarU);

    final NormalDistribution standardNormal = new NormalDistribution(0, 1);

    return 2 * standardNormal.cumulativeProbability(z);
}

Fixed code:
private double calculateAsymptoticPValue(final double Umin,
                                         final int n1,
                                         final int n2)
    throws ConvergenceException, MaxCountExceededException {

    final double n1n2prod = n1 * n2;

    // http://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U#Normal_approximation
    final double EU = n1n2prod / 2.0;
    final double VarU = n1n2prod * (n1 + n2 + 1) / 12.0;

    final double z = (Umin - EU) / FastMath.sqrt(VarU);

    final NormalDistribution standardNormal = new NormalDistribution(0, 1);

    return 2 * standardNormal.cumulativeProbability(z);
}
File: src/main/java/org/apache/commons/math3/optimization/linear/SimplexTableau.java
Issue: SimplexSolver gives bad results
Methode SimplexSolver.optimeze(...) gives bad results with commons-math3-3.0
in a simple test problem. It works well in commons-math-2.2.

Buggy code:
protected void dropPhase1Objective() {
    if (getNumObjectiveFunctions() == 1) {
        return;
    }

    List<Integer> columnsToDrop = new ArrayList<Integer>();
    columnsToDrop.add(0);

    // positive cost non-artificial variables
    for (int i = getNumObjectiveFunctions(); i < getArtificialVariableOffset(); i++) {
        final double entry = tableau.getEntry(0, i);
        if (Precision.compareTo(entry, 0d, maxUlps) > 0) {
            columnsToDrop.add(i);
        }
    }

    // non-basic artificial variables
    for (int i = 0; i < getNumArtificialVariables(); i++) {
      int col = i + getArtificialVariableOffset();
      if (getBasicRow(col) == null) {
        columnsToDrop.add(col);
      }
    }

    double[][] matrix = new double[getHeight() - 1][getWidth() - columnsToDrop.size()];
    for (int i = 1; i < getHeight(); i++) {
      int col = 0;
      for (int j = 0; j < getWidth(); j++) {
        if (!columnsToDrop.contains(j)) {
          matrix[i - 1][col++] = tableau.getEntry(i, j);
        }
      }
    }

    for (int i = columnsToDrop.size() - 1; i >= 0; i--) {
      columnLabels.remove((int) columnsToDrop.get(i));
    }

    this.tableau = new Array2DRowRealMatrix(matrix);
    this.numArtificialVariables = 0;
}

Fixed code:
protected void dropPhase1Objective() {
    if (getNumObjectiveFunctions() == 1) {
        return;
    }

    List<Integer> columnsToDrop = new ArrayList<Integer>();
    columnsToDrop.add(0);

    // positive cost non-artificial variables
    for (int i = getNumObjectiveFunctions(); i < getArtificialVariableOffset(); i++) {
        final double entry = tableau.getEntry(0, i);
        if (Precision.compareTo(entry, 0d, epsilon) > 0) {
            columnsToDrop.add(i);
        }
    }

    // non-basic artificial variables
    for (int i = 0; i < getNumArtificialVariables(); i++) {
      int col = i + getArtificialVariableOffset();
      if (getBasicRow(col) == null) {
        columnsToDrop.add(col);
      }
    }

    double[][] matrix = new double[getHeight() - 1][getWidth() - columnsToDrop.size()];
    for (int i = 1; i < getHeight(); i++) {
      int col = 0;
      for (int j = 0; j < getWidth(); j++) {
        if (!columnsToDrop.contains(j)) {
          matrix[i - 1][col++] = tableau.getEntry(i, j);
        }
      }
    }

    for (int i = columnsToDrop.size() - 1; i >= 0; i--) {
      columnLabels.remove((int) columnsToDrop.get(i));
    }

    this.tableau = new Array2DRowRealMatrix(matrix);
    this.numArtificialVariables = 0;
}
File: src/main/java/org/apache/commons/math3/genetics/ListPopulation.java
Issue: ListPopulation Iterator allows you to remove chromosomes from the population.
Calling the iterator method of ListPopulation returns an iterator of the protected modifiable list. Before returning the iterator we should wrap it in an unmodifiable list.

Buggy code:
public Iterator<Chromosome> iterator() {
    return chromosomes.iterator();
}

Fixed code:
public Iterator<Chromosome> iterator() {
    return getChromosomes().iterator();
}
File: src/main/java/org/apache/commons/math/stat/descriptive/moment/Variance.java
Issue: One of Variance.evaluate() methods does not work correctly
The method org.apache.commons.math.stat.descriptive.moment.Variance.evaluate(double[] values, double[] weights, double mean, int begin, int length) does not work properly. Looks loke it ignores the length parameter and grabs the whole dataset.
Similar method in Mean class seems to work.
I did not check other methods taking the part of the array; they may have the same problem.
Workaround: I had to shrink my arrays and use the method without the length.

Buggy code:
public double evaluate(final double[] values, final double[] weights,
                       final double mean, final int begin, final int length) {

    double var = Double.NaN;

    if (test(values, weights, begin, length)) {
        if (length == 1) {
            var = 0.0;
        } else if (length > 1) {
            double accum = 0.0;
            double dev = 0.0;
            double accum2 = 0.0;
            for (int i = begin; i < begin + length; i++) {
                dev = values[i] - mean;
                accum += weights[i] * (dev * dev);
                accum2 += weights[i] * dev;
            }

            double sumWts = 0;
            for (int i = 0; i < weights.length; i++) {
                sumWts += weights[i];
            }

            if (isBiasCorrected) {
                var = (accum - (accum2 * accum2 / sumWts)) / (sumWts - 1.0);
            } else {
                var = (accum - (accum2 * accum2 / sumWts)) / sumWts;
            }
        }
    }
    return var;
}

Fixed code:
public double evaluate(final double[] values, final double[] weights,
                       final double mean, final int begin, final int length) {

    double var = Double.NaN;

    if (test(values, weights, begin, length)) {
        if (length == 1) {
            var = 0.0;
        } else if (length > 1) {
            double accum = 0.0;
            double dev = 0.0;
            double accum2 = 0.0;
            for (int i = begin; i < begin + length; i++) {
                dev = values[i] - mean;
                accum += weights[i] * (dev * dev);
                accum2 += weights[i] * dev;
            }

            double sumWts = 0;
            for (int i = begin; i < begin + length; i++) {
                sumWts += weights[i];
            }

            if (isBiasCorrected) {
                var = (accum - (accum2 * accum2 / sumWts)) / (sumWts - 1.0);
            } else {
                var = (accum - (accum2 * accum2 / sumWts)) / sumWts;
            }
        }
    }
    return var;
}
File: src/main/java/org/apache/commons/math/stat/descriptive/SummaryStatistics.java
Issue: Statistics.setVarianceImpl makes getStandardDeviation produce NaN
Invoking SummaryStatistics.setVarianceImpl(new Variance(true/false) makes getStandardDeviation produce NaN. The code to reproduce it:

int[] scores = {1, 2, 3, 4};
SummaryStatistics stats = new SummaryStatistics();
stats.setVarianceImpl(new Variance(false)); //use "population variance"
for(int i : scores) {
  stats.addValue(i);
}
double sd = stats.getStandardDeviation();
System.out.println(sd);

A workaround suggested by Mikkel is:

  double sd = FastMath.sqrt(stats.getSecondMoment() / stats.getN());

Buggy code:
public void addValue(double value) {
    sumImpl.increment(value);
    sumsqImpl.increment(value);
    minImpl.increment(value);
    maxImpl.increment(value);
    sumLogImpl.increment(value);
    secondMoment.increment(value);
    // If mean, variance or geomean have been overridden,
    // need to increment these
    if (!(meanImpl instanceof Mean)) {
        meanImpl.increment(value);
    }
    if (!(varianceImpl instanceof Variance)) {
        varianceImpl.increment(value);
    }
    if (!(geoMeanImpl instanceof GeometricMean)) {
        geoMeanImpl.increment(value);
    }
    n++;
}

Fixed code:
public void addValue(double value) {
    sumImpl.increment(value);
    sumsqImpl.increment(value);
    minImpl.increment(value);
    maxImpl.increment(value);
    sumLogImpl.increment(value);
    secondMoment.increment(value);
    // If mean, variance or geomean have been overridden,
    // need to increment these
    if (meanImpl != mean) {
        meanImpl.increment(value);
    }
    if (varianceImpl != variance) {
        varianceImpl.increment(value);
    }
    if (geoMeanImpl != geoMean) {
        geoMeanImpl.increment(value);
    }
    n++;
}
File: src/main/java/org/apache/commons/math/linear/OpenMapRealMatrix.java
Issue: Integer overflow in OpenMapRealMatrix
computeKey() has an integer overflow. Since it is a sparse matrix, this is quite easily encountered long before heap space is exhausted. The attached code demonstrates the problem, which could potentially be a security vulnerability (for example, if one was to use this matrix to store access control information).
Workaround: never create an OpenMapRealMatrix with more cells than are addressable with an int.

Buggy code:
public OpenMapRealMatrix(int rowDimension, int columnDimension) {
    super(rowDimension, columnDimension);
    this.rows = rowDimension;
    this.columns = columnDimension;
    this.entries = new OpenIntToDoubleHashMap(0.0);
}

Fixed code:
public OpenMapRealMatrix(int rowDimension, int columnDimension) {
    super(rowDimension, columnDimension);
    long lRow = (long) rowDimension;
    long lCol = (long) columnDimension;
    if (lRow * lCol >= (long) Integer.MAX_VALUE) {
        throw new NumberIsTooLargeException(lRow * lCol, Integer.MAX_VALUE, false);
    }
    this.rows = rowDimension;
    this.columns = columnDimension;
    this.entries = new OpenIntToDoubleHashMap(0.0);
}
File: src/main/java/org/apache/commons/math3/complex/Complex.java
Issue: Complex.ZERO.reciprocal() returns NaN but should return INF.
Complex.ZERO.reciprocal() returns NaN but should return INF.
Class: org.apache.commons.math3.complex.Complex;
Method: reciprocal()
@version $Id: Complex.java 1416643 2012-12-03 19:37:14Z tn $

Buggy code:
public Complex reciprocal() {
    if (isNaN) {
        return NaN;
    }

    if (real == 0.0 && imaginary == 0.0) {
        return NaN;
    }

    if (isInfinite) {
        return ZERO;
    }

    if (FastMath.abs(real) < FastMath.abs(imaginary)) {
        double q = real / imaginary;
        double scale = 1. / (real * q + imaginary);
        return createComplex(scale * q, -scale);
    } else {
        double q = imaginary / real;
        double scale = 1. / (imaginary * q + real);
        return createComplex(scale, -scale * q);
    }
}

Fixed code:
public Complex reciprocal() {
    if (isNaN) {
        return NaN;
    }

    if (real == 0.0 && imaginary == 0.0) {
        return INF;
    }

    if (isInfinite) {
        return ZERO;
    }

    if (FastMath.abs(real) < FastMath.abs(imaginary)) {
        double q = real / imaginary;
        double scale = 1. / (real * q + imaginary);
        return createComplex(scale * q, -scale);
    } else {
        double q = imaginary / real;
        double scale = 1. / (imaginary * q + real);
        return createComplex(scale, -scale * q);
    }
}
File: src/main/java/org/apache/commons/math/complex/Complex.java
Issue: Complex Add and Subtract handle NaN arguments differently, but javadoc contracts are the same
For both Complex add and subtract, the javadoc states that

     * If either this or <code>rhs</code> has a NaN value in either part,
     * {@link #NaN} is returned; otherwise Inifinite and NaN values are
     * returned in the parts of the result according to the rules for
     * {@link java.lang.Double} arithmetic

Subtract includes an isNaN test and returns Complex.NaN if either complex argument isNaN; but add omits this test.  The test should be added to the add implementation (actually restored, since this looks like a code merge problem going back to 1.1).

Buggy code:
public Complex add(Complex rhs)
    throws NullArgumentException {
    MathUtils.checkNotNull(rhs);
    return createComplex(real + rhs.getReal(),
        imaginary + rhs.getImaginary());
}

Fixed code:
public Complex add(Complex rhs)
    throws NullArgumentException {
    MathUtils.checkNotNull(rhs);
    if (isNaN || rhs.isNaN) {
        return NaN;
    }
    return createComplex(real + rhs.getReal(),
        imaginary + rhs.getImaginary());
}
File: src/main/java/org/apache/commons/math/geometry/Vector3D.java
Issue: Vector3D.crossProduct is sensitive to numerical cancellation
Cross product implementation uses the naive formulas (y1 z2 - y2 z1, ...). These formulas fail when vectors are almost colinear, like in the following example:

Vector3D v1 = new Vector3D(9070467121.0, 4535233560.0, 1);
Vector3D v2 = new Vector3D(9070467123.0, 4535233561.0, 1);
System.out.println(Vector3D.crossProduct(v1, v2));

The previous code displays 
{ -1, 2, 0 }
 instead of the correct answer 
{ -1, 2, 1 }

Buggy code:
public static Vector3D crossProduct(final Vector3D v1, final Vector3D v2) {


    // rescale both vectors without losing precision,
    // to ensure their norm are the same order of magnitude

    // we reduce cancellation errors by preconditioning,
    // we replace v1 by v3 = v1 - rho v2 with rho chosen in order to compute
    // v3 without loss of precision. See Kahan lecture
    // "Computing Cross-Products and Rotations in 2- and 3-Dimensional Euclidean Spaces"
    // available at http://www.cs.berkeley.edu/~wkahan/MathH110/Cross.pdf

    // compute rho as an 8 bits approximation of v1.v2 / v2.v2


    // compute cross product from v3 and v2 instead of v1 and v2
    return new Vector3D(v1.y * v2.z - v1.z * v2.y, v1.z * v2.x - v1.x * v2.z, v1.x * v2.y - v1.y * v2.x);

}

Fixed code:
public static Vector3D crossProduct(final Vector3D v1, final Vector3D v2) {

    final double n1 = v1.getNormSq();
    final double n2 = v2.getNormSq();
    if ((n1 * n2) < MathUtils.SAFE_MIN) {
        return ZERO;
    }

    // rescale both vectors without losing precision,
    // to ensure their norm are the same order of magnitude
    final int deltaExp = (FastMath.getExponent(n1) - FastMath.getExponent(n2)) / 4;
    final double x1    = FastMath.scalb(v1.x, -deltaExp);
    final double y1    = FastMath.scalb(v1.y, -deltaExp);
    final double z1    = FastMath.scalb(v1.z, -deltaExp);
    final double x2    = FastMath.scalb(v2.x,  deltaExp);
    final double y2    = FastMath.scalb(v2.y,  deltaExp);
    final double z2    = FastMath.scalb(v2.z,  deltaExp);

    // we reduce cancellation errors by preconditioning,
    // we replace v1 by v3 = v1 - rho v2 with rho chosen in order to compute
    // v3 without loss of precision. See Kahan lecture
    // "Computing Cross-Products and Rotations in 2- and 3-Dimensional Euclidean Spaces"
    // available at http://www.cs.berkeley.edu/~wkahan/MathH110/Cross.pdf

    // compute rho as an 8 bits approximation of v1.v2 / v2.v2
    final double ratio = (x1 * x2 + y1 * y2 + z1 * z2) / FastMath.scalb(n2, 2 * deltaExp);
    final double rho   = FastMath.rint(256 * ratio) / 256;

    final double x3 = x1 - rho * x2;
    final double y3 = y1 - rho * y2;
    final double z3 = z1 - rho * z2;

    // compute cross product from v3 and v2 instead of v1 and v2
    return new Vector3D(y3 * z2 - z3 * y2, z3 * x2 - x3 * z2, x3 * y2 - y3 * x2);

}
File: src/main/java/org/apache/commons/math/util/FastMath.java
Issue: FastMath.max(50.0f, -50.0f) => -50.0f; should be +50.0f
FastMath.max(50.0f, -50.0f) => -50.0f; should be +50.0f.
This is because the wrong variable is returned.
The bug was not detected by the test case "testMinMaxFloat()" because that has a bug too - it tests doubles, not floats.

Buggy code:
public static float max(final float a, final float b) {
    return (a <= b) ? b : (Float.isNaN(a + b) ? Float.NaN : b);
}

Fixed code:
public static float max(final float a, final float b) {
    return (a <= b) ? b : (Float.isNaN(a + b) ? Float.NaN : a);
}
File: src/main/java/org/apache/commons/math/distribution/NormalDistributionImpl.java
Issue: ConvergenceException in NormalDistributionImpl.cumulativeProbability()
I get a ConvergenceException in  NormalDistributionImpl.cumulativeProbability() for very large/small parameters including Infinity, -Infinity.
For instance in the following code:
	@Test
	public void testCumulative() {
		final NormalDistribution nd = new NormalDistributionImpl();
		for (int i = 0; i < 500; i++) {
			final double val = Math.exp;
			try 
{
				System.out.println("val = " + val + " cumulative = " + nd.cumulativeProbability(val));
			}
 catch (MathException e) 
{
				e.printStackTrace();
				fail();
			}
		}
	}
In version 2.0, I get no exception. 
My suggestion is to change in the implementation of cumulativeProbability(double) to catch all ConvergenceException (and return for very large and very small values), not just MaxIterationsExceededException.

Buggy code:
public double cumulativeProbability(double x) throws MathException {
    final double dev = x - mean;
    try {
    return 0.5 * (1.0 + Erf.erf((dev) /
                (standardDeviation * FastMath.sqrt(2.0))));
    } catch (MaxIterationsExceededException ex) {
        if (x < (mean - 20 * standardDeviation)) { // JDK 1.5 blows at 38
            return 0;
        } else if (x > (mean + 20 * standardDeviation)) {
            return 1;
        } else {
            throw ex;
        }
    }
}

Fixed code:
public double cumulativeProbability(double x) throws MathException {
    final double dev = x - mean;
    if (FastMath.abs(dev) > 40 * standardDeviation) { 
        return dev < 0 ? 0.0d : 1.0d;
    }
    return 0.5 * (1.0 + Erf.erf((dev) /
                (standardDeviation * FastMath.sqrt(2.0))));
}