barty/D4J-Repair huggingface dataset:
File: src/main/java/org/apache/commons/codec/binary/StringUtils.java
Issue: StringUtils.newStringxxx(null) should return null, not NPE
Method calls such as StringUtils.newStringIso8859_1(null) should return null, not NPE.
It looks like this capability was lost with the fix for CODEC-136, i.e.
http://svn.apache.org/viewvc?rev=1306366&view=rev
Several methods were changed from

return StringUtils.newString(bytes, CharEncoding.xxx);
to
return new String(bytes, Charsets.xxx);

The new code should have been:

return newString(bytes, Charsets.xxx);

The newString method handles null input.
There were no tests for null input so the change in behaviour was missed.

Buggy code:
public static String newStringIso8859_1(final byte[] bytes) {
    return new String(bytes, Charsets.ISO_8859_1);
}

Fixed code:
public static String newStringIso8859_1(final byte[] bytes) {
    return newString(bytes, Charsets.ISO_8859_1);
}
File: src/main/java/org/apache/commons/codec/binary/StringUtils.java
Issue: StringUtils.equals(CharSequence cs1, CharSequence cs2) can fail with String Index OBE
StringUtils.equals(CharSequence cs1, CharSequence cs2) fails with String Index OBE if the two sequences are different lengths.

Buggy code:
public static boolean equals(final CharSequence cs1, final CharSequence cs2) {
    if (cs1 == cs2) {
        return true;
    }
    if (cs1 == null || cs2 == null) {
        return false;
    }
    if (cs1 instanceof String && cs2 instanceof String) {
        return cs1.equals(cs2);
    }
    return CharSequenceUtils.regionMatches(cs1, false, 0, cs2, 0, Math.max(cs1.length(), cs2.length()));
}

Fixed code:
public static boolean equals(final CharSequence cs1, final CharSequence cs2) {
    if (cs1 == cs2) {
        return true;
    }
    if (cs1 == null || cs2 == null) {
        return false;
    }
    if (cs1 instanceof String && cs2 instanceof String) {
        return cs1.equals(cs2);
    }
    return cs1.length() == cs2.length() && CharSequenceUtils.regionMatches(cs1, false, 0, cs2, 0, cs1.length());
}
File: src/java/org/apache/commons/codec/binary/Base64.java
Issue: Base64.encodeBase64String() shouldn't chunk
Base64.encodeBase64String() shouldn't chunk.
Change this:

public static String encodeBase64String(byte[] binaryData) {
    return StringUtils.newStringUtf8(encodeBase64(binaryData, true));
}

To this:

public static String encodeBase64String(byte[] binaryData) {
    return StringUtils.newStringUtf8(encodeBase64(binaryData, false));
}

This will fix the following tests ggregory added a few minutes ago:
        //assertEquals("Zg==", Base64.encodeBase64String(StringUtils.getBytesUtf8("f")));
        //assertEquals("Zm8=", Base64.encodeBase64String(StringUtils.getBytesUtf8("fo")));
        //assertEquals("Zm9v", Base64.encodeBase64String(StringUtils.getBytesUtf8("foo")));
        //assertEquals("Zm9vYg==", Base64.encodeBase64String(StringUtils.getBytesUtf8("foob")));
        //assertEquals("Zm9vYmE=", Base64.encodeBase64String(StringUtils.getBytesUtf8("fooba")));
        //assertEquals("Zm9vYmFy", Base64.encodeBase64String(StringUtils.getBytesUtf8("foobar")));

Buggy code:
public static String encodeBase64String(byte[] binaryData) {
    return StringUtils.newStringUtf8(encodeBase64(binaryData, true));
}

Fixed code:
public static String encodeBase64String(byte[] binaryData) {
    return StringUtils.newStringUtf8(encodeBase64(binaryData, false));
}
File: src/java/org/apache/commons/codec/binary/Base64.java
Issue: Base64.encodeBase64(byte[] binaryData, boolean isChunked, boolean urlSafe, int maxResultSize) throws IAE for valid maxResultSize if isChunked is false
If isChunked is false, Base64.encodeBase64(byte[] binaryData, boolean isChunked, boolean urlSafe, int maxResultSize) throws IAE for valid maxResultSize.
Test case and fix will be applied shortly.

Buggy code:
public static byte[] encodeBase64(byte[] binaryData, boolean isChunked, boolean urlSafe, int maxResultSize) {
    if (binaryData == null || binaryData.length == 0) {
        return binaryData;
    }

    long len = getEncodeLength(binaryData, MIME_CHUNK_SIZE, CHUNK_SEPARATOR);
    if (len > maxResultSize) {
        throw new IllegalArgumentException("Input array too big, the output array would be bigger (" +
            len +
            ") than the specified maxium size of " +
            maxResultSize);
    }
            
    Base64 b64 = isChunked ? new Base64(urlSafe) : new Base64(0, CHUNK_SEPARATOR, urlSafe);
    return b64.encode(binaryData);
}

Fixed code:
public static byte[] encodeBase64(byte[] binaryData, boolean isChunked, boolean urlSafe, int maxResultSize) {
    if (binaryData == null || binaryData.length == 0) {
        return binaryData;
    }

    long len = getEncodeLength(binaryData, isChunked ? MIME_CHUNK_SIZE : 0, CHUNK_SEPARATOR);
    if (len > maxResultSize) {
        throw new IllegalArgumentException("Input array too big, the output array would be bigger (" +
            len +
            ") than the specified maxium size of " +
            maxResultSize);
    }
            
    Base64 b64 = isChunked ? new Base64(urlSafe) : new Base64(0, CHUNK_SEPARATOR, urlSafe);
    return b64.encode(binaryData);
}
File: src/main/java/org/apache/commons/collections4/keyvalue/MultiKey.java
Issue: MultiKey subclassing has deserialization problem since COLLECTIONS-266: either declare protected readResolve() or MultiKey must be final
MultiKey from collections 4 provides a transient hashCode and a private readResolve to resolve COLLECTIONS-266: Issue with MultiKey when serialized/deserialized via RMI.
Unfortunately the solution does not work in case of subclassing: readResolve in MultiKey should be declared protected readResolve() to be called during deserialization of the subclass. Otherwise MultiKey must be final to avoid such subclassing.
Testcase:
MultiKeySerializationTest.java
package de.ivu.test.common.collections4;

import static org.junit.Assert.assertEquals;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

import org.apache.commons.collections4.keyvalue.MultiKey;
import org.junit.Test;

public class MultiKeySerializationTest {

    @Test
    @SuppressWarnings("unchecked")
    public void testReadResolveEqualHashCode()
            throws IOException, ClassNotFoundException {
        class MultiKey2<A, B>
                extends MultiKey {

            private static final long serialVersionUID = 1928896152249821416L;

            public MultiKey2(A key1, B key2) {
                super(key1, key2);
            }

            public A getFirst() {
                return (A) getKey(0);
            }

            public B getSecond() {
                return (B) getKey(1);
            }

            // FIXME: MultiKey should either declare protected readResolve() or must be final.
        }
        MultiKey2<String, String> one = new MultiKey2<>("bla", "blub");
        System.out.println(one.hashCode());
        ByteArrayOutputStream byteOut = new ByteArrayOutputStream();
        ObjectOutputStream out = new ObjectOutputStream(byteOut);
        out.writeObject(one);
        out.close();
        byte[] serialized = byteOut.toByteArray();
        ByteArrayInputStream byteIn = new ByteArrayInputStream(serialized);
        ObjectInputStream in = new ObjectInputStream(byteIn);
        MultiKey2<String, String> two = (MultiKey2<String, String>) in.readObject();
        System.out.println(two.hashCode());
        assertEquals("hashCode must be equal - please check for protected readResolve in MultiKey*", one.hashCode(),
            two.hashCode());
    }
}

Fix:
MultiKey.java
@@ -274,7 +274,7 @@
      * only stable for the same process).
      * @return the instance with recalculated hash code
      */
-    private Object readResolve() {
+    protected Object readResolve() {
         calculateHashCode(keys);
         return this;
     }

Buggy code:
private Object readResolve() {
    calculateHashCode(keys);
    return this;
}

Fixed code:
protected Object readResolve() {
    calculateHashCode(keys);
    return this;
}
File: src/main/java/org/apache/commons/compress/archivers/cpio/CpioArchiveOutputStream.java
Issue: CPIO reports unexpected EOF
When unpacking an CPIO archive (made with the compress classes or even made with OSX cpio comandline tool) an EOF exception is thrown.
Here is the testcode:
        final File input = getFile("cmdcreated.cpio");
        final InputStream in = new FileInputStream(input);
        CpioArchiveInputStream cin = new CpioArchiveInputStream(in);
        CpioArchiveEntry entry = null;
        while ((entry = (CpioArchiveEntry) cin.getNextCPIOEntry()) != null) 
{
            File target = new File(dir, entry.getName());
            final OutputStream out = new FileOutputStream(target);
            IOUtils.copy(in, out);
            out.close();
        }

        cin.close();
Stacktrace is here:
java.io.EOFException
	at org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.readFully(CpioArchiveInputStream.java:293)
	at org.apache.commons.compress.archivers.cpio.CpioArchiveInputStream.getNextCPIOEntry(CpioArchiveInputStream.java:168)
	at org.apache.commons.compress.archivers.cpio.CpioArchiveInputStreamTest.testCpioUnpack(CpioArchiveInputStreamTest.java:26)
	...
This happens with the first read access to the archive. It occured while my try to improve the testcases.

Buggy code:
public void close() throws IOException {
    if (!this.closed) {
        super.close();
        this.closed = true;
    }
}

Fixed code:
public void close() throws IOException {
    if (!this.closed) {
        this.finish();
        super.close();
        this.closed = true;
    }
}
File: src/main/java/org/apache/commons/compress/archivers/zip/ZipArchiveEntry.java
Issue: ArchiveInputStream#getNextEntry(): Problems with WinZip directories with Umlauts
There is a problem when handling a WinZip-created zip with Umlauts in directories.
I'm accessing a zip file created with WinZip containing a directory with an umlaut ("ä") with ArchiveInputStream. When creating the zip file the unicode-flag of winzip had been active.
The following problem occurs when accessing the entries of the zip:
the ArchiveEntry for a directory containing an umlaut is not marked as a directory and the file names for the directory and all files contained in that directory contain backslashes instead of slashes (i.e. completely different to all other files in directories with no umlaut in their path).
There is no difference when letting the ArchiveStreamFactory decide which ArchiveInputStream to create or when using the ZipArchiveInputStream constructor with the correct encoding (I've tried different encodings CP437, CP850, ISO-8859-15, but still the problem persisted).
This problem does not occur when using the very same zip file but compressed by 7zip or the built-in Windows 7 zip functionality.

Buggy code:
protected void setName(String name) {
    this.name = name;
}

Fixed code:
protected void setName(String name) {
    if (name != null && getPlatform() == PLATFORM_FAT
        && name.indexOf("/") == -1) {
        name = name.replace('\\', '/');
    }
    this.name = name;
}
File: src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZOutputFile.java
Issue: Writing 7z empty entries produces incorrect or corrupt archive
I couldn't find an exact rule that causes this incorrect behavior, but I tried to reduce it to some simple scenarios to reproduce it:
Input: A folder with certain files -> tried to archive it.
If the folder contains more than 7 files the incorrect behavior appears.
Scenario 1: 7 empty files
Result: The created archive contains a single folder entry with the name of the archive (no matter which was the name of the file)
Scenario 2: 7 files, some empty, some with content
Result: The created archive contains a folder entry with the name of the archive and a number of file entries also with the name of the archive. The number of the entries is equal to the number of non empty files.
Scenario 3: 8 empty files
Result: 7zip Manager cannot open archive and stops working.
Scenario 4.1: 8 files: some empty, some with content, last file (alphabetically) with content
Result: same behavior as described for Scenario 2.
Scenario 4.2: 8 files, some empty, some with content, last file empy
Result: archive is corrupt, the following message is received: "Cannot open file 'archivename.7z' as archive" (7Zip Manager does not crash).

Buggy code:
private void writeBits(final DataOutput header, final BitSet bits, final int length) throws IOException {
    int cache = 0;
    int shift = 7;
    for (int i = 0; i < length; i++) {
        cache |= ((bits.get(i) ? 1 : 0) << shift);
        --shift;
        if (shift == 0) {
            header.write(cache);
            shift = 7;
            cache = 0;
        }
    }
    if (length > 0 && shift > 0) {
        header.write(cache);
    }
}

Fixed code:
private void writeBits(final DataOutput header, final BitSet bits, final int length) throws IOException {
    int cache = 0;
    int shift = 7;
    for (int i = 0; i < length; i++) {
        cache |= ((bits.get(i) ? 1 : 0) << shift);
        if (--shift < 0) {
            header.write(cache);
            shift = 7;
            cache = 0;
        }
    }
    if (shift != 7) {
        header.write(cache);
    }
}
File: src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java
Issue: 7z: 16 MB dictionary is too big
I created an archiv with 7zip 9.20 containing the compress-1.7-src directory. Also tried it with 1.6 version and directory. I 
downloaded the zip file and reziped it as 7z. The standard setting where used:
Compression level: normal
Compression method: lzma2
Dictionary size: 16 MB
Word size: 32
Solid Block size: 2 GB
I get an exception if I try to open the file with the simple line of code:
SevenZFile input = new SevenZFile(new File(arcName));
Maybe it is a bug in the tukaani library, but I do not know how to report it to them.
The exception thrown:
org.tukaani.xz.UnsupportedOptionsException: LZMA dictionary is too big for this implementation
	at org.tukaani.xz.LZMAInputStream.initialize(Unknown Source)
	at org.tukaani.xz.LZMAInputStream.<init>(Unknown Source)
	at org.apache.commons.compress.archivers.sevenz.Coders$LZMADecoder.decode(Coders.java:117)
	at org.apache.commons.compress.archivers.sevenz.Coders.addDecoder(Coders.java:48)
	at org.apache.commons.compress.archivers.sevenz.SevenZFile.readEncodedHeader(SevenZFile.java:278)
	at org.apache.commons.compress.archivers.sevenz.SevenZFile.readHeaders(SevenZFile.java:190)
	at org.apache.commons.compress.archivers.sevenz.SevenZFile.<init>(SevenZFile.java:94)
	at org.apache.commons.compress.archivers.sevenz.SevenZFile.<init>(SevenZFile.java:116)
	at compress.SevenZipError.main(SevenZipError.java:28)

Buggy code:
InputStream decode(final InputStream in, final Coder coder,
        byte[] password) throws IOException {
    byte propsByte = coder.properties[0];
    long dictSize = coder.properties[1];
    for (int i = 1; i < 4; i++) {
        dictSize |= (coder.properties[i + 1] << (8 * i));
    }
    if (dictSize > LZMAInputStream.DICT_SIZE_MAX) {
        throw new IOException("Dictionary larger than 4GiB maximum size");
    }
    return new LZMAInputStream(in, -1, propsByte, (int) dictSize);
}

Fixed code:
InputStream decode(final InputStream in, final Coder coder,
        byte[] password) throws IOException {
    byte propsByte = coder.properties[0];
    long dictSize = coder.properties[1];
    for (int i = 1; i < 4; i++) {
        dictSize |= (coder.properties[i + 1] & 0xffl) << (8 * i);
    }
    if (dictSize > LZMAInputStream.DICT_SIZE_MAX) {
        throw new IOException("Dictionary larger than 4GiB maximum size");
    }
    return new LZMAInputStream(in, -1, propsByte, (int) dictSize);
}
File: src/main/java/org/apache/commons/compress/utils/IOUtils.java
Issue: IOUtils.skip does not work as advertised
I am trying to feed a TarInputStream from a CipherInputStream.
It does not work, because IOUtils.skip() does not adhere to the contract it claims in javadoc:
"     * <p>This method will only skip less than the requested number of

bytes if the end of the input stream has been reached.</p>"

However it does:
            long skipped = input.skip(numToSkip);
            if (skipped == 0) 
{
                break;
            }

And the input stream javadoc says:
"     * This may result from any of a number of conditions; reaching end of file

before <code>n</code> bytes have been skipped is only one possibility."

In the case of CipherInputStream, it stops at the end of each byte buffer.
If you check the IOUtils from colleagues at commons-io, they have considered this case in IOUtils.skip() where they use a read to skip through the stream.
An optimized version could combine trying to skip, then read then trying to skip again.

Buggy code:
public static long skip(InputStream input, long numToSkip) throws IOException {
    long available = numToSkip;
    while (numToSkip > 0) {
        long skipped = input.skip(numToSkip);
        if (skipped == 0) {
            break;
        }
        numToSkip -= skipped;
    }
        
    return available - numToSkip;
}

Fixed code:
public static long skip(InputStream input, long numToSkip) throws IOException {
    long available = numToSkip;
    while (numToSkip > 0) {
        long skipped = input.skip(numToSkip);
        if (skipped == 0) {
            break;
        }
        numToSkip -= skipped;
    }
        
    if (numToSkip > 0) {
        byte[] skipBuf = new byte[SKIP_BUF_SIZE];
        while (numToSkip > 0) {
            int read = readFully(input, skipBuf, 0,
                                 (int) Math.min(numToSkip, SKIP_BUF_SIZE));
            if (read < 1) {
                break;
            }
            numToSkip -= read;
        }
    }
    return available - numToSkip;
}
File: src/main/java/org/apache/commons/compress/archivers/tar/TarArchiveInputStream.java
Issue: TarArchiveInputStream silently finished when unexpected EOF occured
I just found the following test case didn't raise an IOException as it used to be for a tar trimmed on purpose 
@Test
  public void testCorruptedBzip2() throws IOException {
    String archivePath = PathUtil.join(testdataDir, "test.tar.bz2");
    TarArchiveInputStream input = null;
    input = new TarArchiveInputStream(new BZip2CompressorInputStream(
        GoogleFile.SYSTEM.newInputStream(archivePath), true));
    ArchiveEntry nextMatchedEntry = input.getNextEntry();
    while (nextMatchedEntry != null) 
{
      logger.infofmt("Extracting %s", nextMatchedEntry.getName());
      String outputPath = PathUtil.join("/tmp/", nextMatchedEntry.getName());
      OutputStream out = new FileOutputStream(outputPath);
      ByteStreams.copy(input, out);
      out.close();
      nextMatchedEntry = input.getNextEntry();
    }
  }

Buggy code:
public int read(byte[] buf, int offset, int numToRead) throws IOException {
	int totalRead = 0;

    if (hasHitEOF || entryOffset >= entrySize) {
        return -1;
    }

    if (currEntry == null) {
        throw new IllegalStateException("No current tar entry");
    }

    numToRead = Math.min(numToRead, available());
    
    totalRead = is.read(buf, offset, numToRead);
    count(totalRead);
    
    if (totalRead == -1) {
        hasHitEOF = true;
    } else {
        entryOffset += totalRead;
    }

    return totalRead;
}

Fixed code:
public int read(byte[] buf, int offset, int numToRead) throws IOException {
	int totalRead = 0;

    if (hasHitEOF || entryOffset >= entrySize) {
        return -1;
    }

    if (currEntry == null) {
        throw new IllegalStateException("No current tar entry");
    }

    numToRead = Math.min(numToRead, available());
    
    totalRead = is.read(buf, offset, numToRead);
    
    if (totalRead == -1) {
        if (numToRead > 0) {
            throw new IOException("Truncated TAR archive");
        }
        hasHitEOF = true;
    } else {
        count(totalRead);
        entryOffset += totalRead;
    }

    return totalRead;
}
File: src/main/java/org/apache/commons/compress/archivers/tar/TarUtils.java
Issue: TAR checksum fails when checksum is right aligned
The linked TAR has a checksum with zero padding on the left instead of the expected NULL-SPACE terminator on the right. As a result the last two digits of the stored checksum are lost and the otherwise valid checksum is treated as invalid.
Given that the code already checks for digits being in range before adding them to the stored sum, is it necessary to only look at the first 6 octal digits instead of the whole field?

Buggy code:
public static boolean verifyCheckSum(byte[] header) {
    long storedSum = 0;
    long unsignedSum = 0;
    long signedSum = 0;

    int digits = 0;
    for (int i = 0; i < header.length; i++) {
        byte b = header[i];
        if (CHKSUM_OFFSET  <= i && i < CHKSUM_OFFSET + CHKSUMLEN) {
            if ('0' <= b && b <= '7' && digits++ < 6) {
                storedSum = storedSum * 8 + b - '0';
            } else if (digits > 0) {
                digits = 6;
            }
            b = ' ';
        }
        unsignedSum += 0xff & b;
        signedSum += b;
    }
    return storedSum == unsignedSum || storedSum == signedSum;
}

Fixed code:
public static boolean verifyCheckSum(byte[] header) {
    long storedSum = parseOctal(header, CHKSUM_OFFSET, CHKSUMLEN);
    long unsignedSum = 0;
    long signedSum = 0;

    int digits = 0;
    for (int i = 0; i < header.length; i++) {
        byte b = header[i];
        if (CHKSUM_OFFSET  <= i && i < CHKSUM_OFFSET + CHKSUMLEN) {
            b = ' ';
        }
        unsignedSum += 0xff & b;
        signedSum += b;
    }
    return storedSum == unsignedSum || storedSum == signedSum;
}
File: src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java
Issue: Calling SevenZFile.read() on empty SevenZArchiveEntry throws IllegalStateException
I'm pretty sure COMPRESS-340 breaks reading empty archive entries. When calling getNextEntry() and that entry has no content, the code jumps into the first block at line 830 (SevenZFile.class), clearing the deferredBlockStreams. When calling entry.read(...) afterwards an IllegalStateException ("No current 7z entry (call getNextEntry() first).") is thrown. IMHO, there should be another check for entry.getSize() == 0.
This worked correctly up until 1.10.

Buggy code:
private InputStream getCurrentStream() throws IOException {
    if (deferredBlockStreams.isEmpty()) {
        throw new IllegalStateException("No current 7z entry (call getNextEntry() first).");
    }
    
    while (deferredBlockStreams.size() > 1) {
        // In solid compression mode we need to decompress all leading folder'
        // streams to get access to an entry. We defer this until really needed
        // so that entire blocks can be skipped without wasting time for decompression.
        final InputStream stream = deferredBlockStreams.remove(0);
        IOUtils.skip(stream, Long.MAX_VALUE);
        stream.close();
    }

    return deferredBlockStreams.get(0);
}

Fixed code:
private InputStream getCurrentStream() throws IOException {
    if (archive.files[currentEntryIndex].getSize() == 0) {
        return new ByteArrayInputStream(new byte[0]);
    }
    if (deferredBlockStreams.isEmpty()) {
        throw new IllegalStateException("No current 7z entry (call getNextEntry() first).");
    }
    
    while (deferredBlockStreams.size() > 1) {
        // In solid compression mode we need to decompress all leading folder'
        // streams to get access to an entry. We defer this until really needed
        // so that entire blocks can be skipped without wasting time for decompression.
        final InputStream stream = deferredBlockStreams.remove(0);
        IOUtils.skip(stream, Long.MAX_VALUE);
        stream.close();
    }

    return deferredBlockStreams.get(0);
}