barty/D4J-Repair huggingface dataset:
File: src/main/java/com/fasterxml/jackson/dataformat/xml/ser/XmlSerializerProvider.java
Issue: XmlSerializerProvider does not use withRootName config for null
In jackson-dataformat-xml/src/main/java/com/fasterxml/jackson/dataformat/xml/ser/XmlSerializerProvider.java
Line 203, I think _rootNameFromConfig() should be used if available instead of ROOT_NAME_FOR_NULL, so that withRootName() config can be used.
I don't know whether/how deser would be affected

jackson-dataformat-xml/src/main/java/com/fasterxml/jackson/dataformat/xml/ser/XmlSerializerProvider.java

         Line 203
      in
      ca1c671

 _initWithRootName((ToXmlGenerator) jgen, ROOT_NAME_FOR_NULL);

Buggy code:
protected void _serializeXmlNull(JsonGenerator jgen) throws IOException
{
    // 14-Nov-2016, tatu: As per [dataformat-xml#213], we may have explicitly
    //    configured root name...
    if (jgen instanceof ToXmlGenerator) {
        _initWithRootName((ToXmlGenerator) jgen, ROOT_NAME_FOR_NULL);
    }
    super.serializeValue(jgen, null);
}

Fixed code:
protected void _serializeXmlNull(JsonGenerator jgen) throws IOException
{
    // 14-Nov-2016, tatu: As per [dataformat-xml#213], we may have explicitly
    //    configured root name...
    QName rootName = _rootNameFromConfig();
    if (rootName == null) {
        rootName = ROOT_NAME_FOR_NULL;
    }
    if (jgen instanceof ToXmlGenerator) {
        _initWithRootName((ToXmlGenerator) jgen, rootName);
    }
    super.serializeValue(jgen, null);
}
File: src/main/java/com/fasterxml/jackson/dataformat/xml/ser/XmlSerializerProvider.java
Issue: @JacksonXmlRootElement malfunction when using it with multiple XmlMappers and disabling annotations
Found this in version 2.9.4 running some tests that go back and forth serializing with an XML mapper that uses annotations, and another one that ignores them. May be related to issue #171 and the cache of class annotations.
When running this code, the second print statement should use the annotation's localName but it instead uses the class name.
@JacksonXmlRootElement(localName = "myname")
public class XMLTest {

    public static void main(String[] s) throws Exception {

        final ObjectMapper xmlMapper = new XmlMapper();
        final ObjectMapper noAnnotationsXmlMapper = xmlMapper.copy()
                .configure(MapperFeature.USE_ANNOTATIONS, false)
                .configure(SerializationFeature.FAIL_ON_EMPTY_BEANS, false);

        System.out.println(noAnnotationsXmlMapper.writeValueAsString(new XMLTest()));
        System.out.println(xmlMapper.writeValueAsString(new XMLTest()));

    }

}

Output:
<XMLTest/>
<XMLTest/>

Buggy code:
protected XmlSerializerProvider(XmlSerializerProvider src) {
    super(src);
    // 21-May-2018, tatu: As per [dataformat-xml#282], should NOT really copy
    //    root name lookup as that may link back to diff version, configuration
    _rootNameLookup = src._rootNameLookup;
}

Fixed code:
protected XmlSerializerProvider(XmlSerializerProvider src) {
    super(src);
    // 21-May-2018, tatu: As per [dataformat-xml#282], should NOT really copy
    //    root name lookup as that may link back to diff version, configuration
    _rootNameLookup = new XmlRootNameLookup();
}
File: src/main/java/org/jsoup/nodes/Document.java
Issue: Parsing a HTML snippet causes the leading text to be moved to back
Code:
String html = "foo <b>bar</b> baz";
String text = Jsoup.parse(html).text();
System.out.println(text);

Result:
bar baz foo

Expected:
foo bar baz

Buggy code:
private void normalise(Element element) {
    List<Node> toMove = new ArrayList<Node>();
    for (Node node: element.childNodes) {
        if (node instanceof TextNode) {
            TextNode tn = (TextNode) node;
            if (!tn.isBlank())
                toMove.add(tn);
        }
    }

    for (Node node: toMove) {
        element.removeChild(node);
        body().appendChild(new TextNode(" ", ""));
        body().appendChild(node);
    }
}

Fixed code:
private void normalise(Element element) {
    List<Node> toMove = new ArrayList<Node>();
    for (Node node: element.childNodes) {
        if (node instanceof TextNode) {
            TextNode tn = (TextNode) node;
            if (!tn.isBlank())
                toMove.add(tn);
        }
    }

    for (Node node: toMove) {
        element.removeChild(node);
        body().prependChild(node);
        body().prependChild(new TextNode(" ", ""));
    }
}
File: src/main/java/org/jsoup/nodes/Node.java
Issue: attr("abs:href") , absUrl("href")
Document doc = Jsoup.parse(new URL("http://www.oschina.net/bbs/thread/12975"), 5*1000);
Elements es = doc.select("a[href]");
for(Iterator it = es.iterator();it.hasNext();){
Element e = it.next();
System.out.println(e.absUrl("href"));
}
attr("abs:href")   ------  <a href="?p=1">1</a>
result: -------------------   http://www.oschina.net/bbs/thread/?p=1
I think it's a wrong result~.
The correct results should be "http://www.oschina.net/bbs/thread/12975?p=1"

Buggy code:
public String absUrl(String attributeKey) {
    Validate.notEmpty(attributeKey);

    String relUrl = attr(attributeKey);
    if (!hasAttr(attributeKey)) {
        return ""; // nothing to make absolute with
    } else {
        URL base;
        try {
            try {
                base = new URL(baseUri);
            } catch (MalformedURLException e) {
                // the base is unsuitable, but the attribute may be abs on its own, so try that
                URL abs = new URL(relUrl);
                return abs.toExternalForm();
            }
            // workaround: java resolves '//path/file + ?foo' to '//path/?foo', not '//path/file?foo' as desired
            URL abs = new URL(base, relUrl);
            return abs.toExternalForm();
        } catch (MalformedURLException e) {
            return "";
        }
    }
}

Fixed code:
public String absUrl(String attributeKey) {
    Validate.notEmpty(attributeKey);

    String relUrl = attr(attributeKey);
    if (!hasAttr(attributeKey)) {
        return ""; // nothing to make absolute with
    } else {
        URL base;
        try {
            try {
                base = new URL(baseUri);
            } catch (MalformedURLException e) {
                // the base is unsuitable, but the attribute may be abs on its own, so try that
                URL abs = new URL(relUrl);
                return abs.toExternalForm();
            }
            // workaround: java resolves '//path/file + ?foo' to '//path/?foo', not '//path/file?foo' as desired
            if (relUrl.startsWith("?"))
                relUrl = base.getPath() + relUrl;
            URL abs = new URL(base, relUrl);
            return abs.toExternalForm();
        } catch (MalformedURLException e) {
            return "";
        }
    }
}
File: src/main/java/org/jsoup/nodes/Node.java
Issue: abs: attribute prefix does not work on Elements.attr()
Elements.attr() iterates on its element to look for the first one with the given attrbute.
If I try to get the attribute abs:href, the test element.hasAttr("abs:herf") fails, and the returned value is an empty string.

Buggy code:
public boolean hasAttr(String attributeKey) {
    Validate.notNull(attributeKey);

    return attributes.hasKey(attributeKey);
}

Fixed code:
public boolean hasAttr(String attributeKey) {
    Validate.notNull(attributeKey);

    if (attributeKey.toLowerCase().startsWith("abs:")) {
        String key = attributeKey.substring("abs:".length());
        if (attributes.hasKey(key) && !absUrl(key).equals(""))
            return true;
    }
    return attributes.hasKey(attributeKey);
}
File: src/main/java/org/jsoup/safety/Whitelist.java
Issue: Cleaning html containing the cid identifier breaks images
Ok, so in mail type HTML the following is common

The item after CID: can be almost anything (US-ASCII I think) and of any length. It corresponds to an image linked elsewhere in MIME say like this
--mimebounday
Content-ID:
Content-Type: image/jpeg.....
(snip)
So, to mark a long story somewhat shorter, I use Jsoup's sanitizer extensively. However, I need these CID references to be preserved post sanitization. addProtocols does not work because the items are not valid URLs. As a result
the above becomes . Which for my purposes is not good :)

Buggy code:
private boolean testValidProtocol(Element el, Attribute attr, Set<Protocol> protocols) {
    // try to resolve relative urls to abs, and optionally update the attribute so output html has abs.
    // rels without a baseuri get removed
    String value = el.absUrl(attr.getKey());
    if (!preserveRelativeLinks)
        attr.setValue(value);
    
    for (Protocol protocol : protocols) {
        String prot = protocol.toString() + ":";
        if (value.toLowerCase().startsWith(prot)) {
            return true;
        }
    }
    return false;
}

Fixed code:
private boolean testValidProtocol(Element el, Attribute attr, Set<Protocol> protocols) {
    // try to resolve relative urls to abs, and optionally update the attribute so output html has abs.
    // rels without a baseuri get removed
    String value = el.absUrl(attr.getKey());
    if (value.length() == 0)
        value = attr.getValue(); // if it could not be made abs, run as-is to allow custom unknown protocols
    if (!preserveRelativeLinks)
        attr.setValue(value);
    
    for (Protocol protocol : protocols) {
        String prot = protocol.toString() + ":";
        if (value.toLowerCase().startsWith(prot)) {
            return true;
        }
    }
    return false;
}
File: src/main/java/org/jsoup/parser/TokeniserState.java
Issue: 1.6.0 dropping a ' on a particular javascript string
Loses a single quote when the javascript contains a partial tag, exampled pared from ad section of http://scienceblogs.com/pharyngula.  Note in the result that '</scr is missing closing ' :
Input:
<HTML>
<body>
 <div>
  <script language="JavaScript1.1"> 
    document.write('</scr' + 'ipt>');
  </script>
 </div>
</body>
</HTML>

Result:
<html>
 <body> 
  <div> 
   <script language="JavaScript1.1"> 
    document.write('</scr + 'ipt>');

   </script> 
  </div>  
 </body>
</html>

Buggy code:
void read(Tokeniser t, CharacterReader r) {
    if (r.matchesLetter()) {
        String name = r.consumeLetterSequence();
        t.tagPending.appendTagName(name.toLowerCase());
        t.dataBuffer.append(name);
        r.advance();
        return;
    }

    if (t.isAppropriateEndTagToken() && !r.isEmpty()) {
        char c = r.consume();
        switch (c) {
            case '\t':
            case '\n':
            case '\f':
            case ' ':
                t.transition(BeforeAttributeName);
                break;
            case '/':
                t.transition(SelfClosingStartTag);
                break;
            case '>':
                t.emitTagPending();
                t.transition(Data);
                break;
            default:
                t.dataBuffer.append(c);
                anythingElse(t, r);
                break;
        }
    } else {
        anythingElse(t, r);
    }
}

Fixed code:
void read(Tokeniser t, CharacterReader r) {
    if (r.matchesLetter()) {
        String name = r.consumeLetterSequence();
        t.tagPending.appendTagName(name.toLowerCase());
        t.dataBuffer.append(name);
        return;
    }

    if (t.isAppropriateEndTagToken() && !r.isEmpty()) {
        char c = r.consume();
        switch (c) {
            case '\t':
            case '\n':
            case '\f':
            case ' ':
                t.transition(BeforeAttributeName);
                break;
            case '/':
                t.transition(SelfClosingStartTag);
                break;
            case '>':
                t.emitTagPending();
                t.transition(Data);
                break;
            default:
                t.dataBuffer.append(c);
                anythingElse(t, r);
                break;
        }
    } else {
        anythingElse(t, r);
    }
}
File: src/main/java/org/jsoup/safety/Cleaner.java
Issue: NullpointerException when applying Cleaner to a frameset
To reproduce:

Create/find a html document of a frameset.
Parse the html.
Create a Cleaner instance and call the clean method with the document from step 2.
NullPointerException

Cause:
In Cleaner.clean(Document) (https://github.com/jhy/jsoup/blob/master/src/main/java/org/jsoup/safety/Cleaner.java#L43) the copySafeNodes is called with the document.body(). However, this is null when handling a frameset document.
Expected:
An empty document or perhaps null returned. But not a nullpointerException.

Buggy code:
public Document clean(Document dirtyDocument) {
    Validate.notNull(dirtyDocument);

    Document clean = Document.createShell(dirtyDocument.baseUri());
        copySafeNodes(dirtyDocument.body(), clean.body());

    return clean;
}

Fixed code:
public Document clean(Document dirtyDocument) {
    Validate.notNull(dirtyDocument);

    Document clean = Document.createShell(dirtyDocument.baseUri());
    if (dirtyDocument.body() != null) // frameset documents won't have a body. the clean doc will have empty body.
        copySafeNodes(dirtyDocument.body(), clean.body());

    return clean;
}
File: src/main/java/org/jsoup/helper/DataUtil.java
Issue: Invalid HTTP-Response header leads to exception
In particular case a HTTP-Webpage responses with a invalid HTTP-Charset field (delivered UFT8 instead of UTF8).
This leads to an UnsupportedCharsetException in org.jsoup.helper.DataUtil at around Line 93(?) where :
  Validate.notEmpty(charsetName, "Must set charset arg to character set of file to parse. Set to null to attempt to detect from HTML");
  docData = Charset.forName(charsetName).decode(byteData).toString();
I fixed it by wrapping a try catch statement around these two lines such that:
try{
  Validate.notEmpty(charsetName, "Must set charset arg to character set of file to parse. Set to null to attempt to detect from HTML");
  docData = Charset.forName(charsetName).decode(byteData).toString();
} catch(UnsupportedCharsetException e){
  return parseByteData(byteData,(String)null,baseUri,parser);
}
It now falls back to the none charset argument assigned clause, and tries to detect the character set via HTML.

Buggy code:
static String getCharsetFromContentType(String contentType) {
    if (contentType == null) return null;
    Matcher m = charsetPattern.matcher(contentType);
    if (m.find()) {
        String charset = m.group(1).trim();
        charset = charset.toUpperCase(Locale.ENGLISH);
        return charset;
    }
    return null;
}

Fixed code:
static String getCharsetFromContentType(String contentType) {
    if (contentType == null) return null;
    Matcher m = charsetPattern.matcher(contentType);
    if (m.find()) {
        String charset = m.group(1).trim();
        if (Charset.isSupported(charset)) return charset;
        charset = charset.toUpperCase(Locale.ENGLISH);
        if (Charset.isSupported(charset)) return charset;
    }
    return null;
}
File: src/main/java/org/jsoup/nodes/Element.java
Issue: Element.clone() wrongly shared a same classNames Set instance
In the clone() method of Node, the Object.clone() is called, if the original element's classNames Set had been initialized before clone, the original classNames Set will be set to the new cloned Element instance due to the JDK's clone mechanism. Thus, the old element and the newly cloned Element will share a same classNames Set instance.

Buggy code:
public Element clone() {
    Element clone = (Element) super.clone();
    clone.classNames();
    return clone;
}

Fixed code:
public Element clone() {
    Element clone = (Element) super.clone();
    clone.classNames = null; // derived on first hit, otherwise gets a pointer to source classnames
    return clone;
}
File: src/main/java/org/jsoup/parser/HtmlTreeBuilder.java
Issue: Self-closing script tag causes remainder of document to be html-escaped.
When a self-closing script block is encountered it appears that the state transitions do not account for the closing tag, so the rest of the document is considered to be in the body of the script tag, and so is escaped.
The unit test HtmlParserTest.handlesKnownEmptyBlocks() will fail if a self-closing script tag is included in the String h.

Buggy code:
Element insert(Token.StartTag startTag) {
    // handle empty unknown tags
    // when the spec expects an empty tag, will directly hit insertEmpty, so won't generate this fake end tag.
    if (startTag.isSelfClosing()) {
        Element el = insertEmpty(startTag);
        stack.add(el);
        tokeniser.emit(new Token.EndTag(el.tagName()));  // ensure we get out of whatever state we are in. emitted for yielded processing
        return el;
    }
    
    Element el = new Element(Tag.valueOf(startTag.name()), baseUri, startTag.attributes);
    insert(el);
    return el;
}

Fixed code:
Element insert(Token.StartTag startTag) {
    // handle empty unknown tags
    // when the spec expects an empty tag, will directly hit insertEmpty, so won't generate this fake end tag.
    if (startTag.isSelfClosing()) {
        Element el = insertEmpty(startTag);
        stack.add(el);
        tokeniser.transition(TokeniserState.Data); // handles <script />, otherwise needs breakout steps from script data
        tokeniser.emit(new Token.EndTag(el.tagName()));  // ensure we get out of whatever state we are in. emitted for yielded processing
        return el;
    }
    
    Element el = new Element(Tag.valueOf(startTag.name()), baseUri, startTag.attributes);
    insert(el);
    return el;
}
File: src/main/java/org/jsoup/parser/CharacterReader.java
Issue: Parser error on commented CDATA
Jsoup gives the following error when trying to parse this HTML: https://gist.github.com/felipehummel/6122799
java.lang.ArrayIndexOutOfBoundsException: 8666
    at org.jsoup.parser.CharacterReader.nextIndexOf(CharacterReader.java:92)
    at org.jsoup.parser.CharacterReader.consumeTo(CharacterReader.java:112)
    at org.jsoup.parser.TokeniserState$67.read(TokeniserState.java:1789)
    at org.jsoup.parser.Tokeniser.read(Tokeniser.java:42)
    at org.jsoup.parser.TreeBuilder.runParser(TreeBuilder.java:47)
    at org.jsoup.parser.TreeBuilder.parse(TreeBuilder.java:41)
    at org.jsoup.parser.HtmlTreeBuilder.parse(HtmlTreeBuilder.java:37)
    at org.jsoup.parser.Parser.parse(Parser.java:90)
    at org.jsoup.Jsoup.parse(Jsoup.java:58)
...

The HTML is from a entry in a RSS feed. If I remove the line:
// ]]

or just the
]]
Then it parses the HTML nicely.
Does this syntax error should really throw an exception or it should be silently ignored?

Buggy code:
int nextIndexOf(CharSequence seq) {
    // doesn't handle scanning for surrogates
    char startChar = seq.charAt(0);
    for (int offset = pos; offset < length; offset++) {
        // scan to first instance of startchar:
        if (startChar != input[offset])
            while(++offset < length && startChar != input[offset]);
        int i = offset + 1;
        int last = i + seq.length()-1;
        if (offset < length) {
            for (int j = 1; i < last && seq.charAt(j) == input[i]; i++, j++);
            if (i == last) // found full sequence
                return offset - pos;
        }
    }
    return -1;
}

Fixed code:
int nextIndexOf(CharSequence seq) {
    // doesn't handle scanning for surrogates
    char startChar = seq.charAt(0);
    for (int offset = pos; offset < length; offset++) {
        // scan to first instance of startchar:
        if (startChar != input[offset])
            while(++offset < length && startChar != input[offset]);
        int i = offset + 1;
        int last = i + seq.length()-1;
        if (offset < length && last <= length) {
            for (int j = 1; i < last && seq.charAt(j) == input[i]; i++, j++);
            if (i == last) // found full sequence
                return offset - pos;
        }
    }
    return -1;
}
File: src/main/java/org/jsoup/nodes/Element.java
Issue: Whitespaces are discared in Element.html() method
Hi,
I'm trying to make an exact copy of a document (changing just a couple of attributes and appending a few nodes) and the trim() inside the Element.html() is killing me.
I'm using Parsers.xml() and no prettyPrint.
I think this trim should be enabled for prettyPrint only.

Buggy code:
public String html() {
    StringBuilder accum = new StringBuilder();
    html(accum);
    return accum.toString().trim();
}

Fixed code:
public String html() {
    StringBuilder accum = new StringBuilder();
    html(accum);
    return getOutputSettings().prettyPrint() ? accum.toString().trim() : accum.toString();
}
File: src/main/java/org/jsoup/nodes/DocumentType.java
Issue: "<!DOCTYPE>" IllegalArgumentException: String must not be empty
While this may be a contrived example, Jsoup.parse("<!DOCTYPE>") throws an exception, this was unexpected. Possibly related, a proper document with <!DOCTYPE> (no name) is generating corrupt html e.g. "<!DOCTYPE <html> ..." (missing right angle bracket on DOCTYPE.)
Spec says "When a DOCTYPE token is created, its name, public identifier, and system identifier must be marked as missing (which is a distinct state from the empty string), [...]"

Buggy code:
public DocumentType(String name, String publicId, String systemId, String baseUri) {
    super(baseUri);

    Validate.notEmpty(name);
    attr("name", name);
    attr("publicId", publicId);
    attr("systemId", systemId);
}

Fixed code:
public DocumentType(String name, String publicId, String systemId, String baseUri) {
    super(baseUri);

    attr("name", name);
    attr("publicId", publicId);
    attr("systemId", systemId);
}
File: src/main/java/org/jsoup/nodes/Element.java
Issue: Element.hashCode() ignores the content text of the element.
Found this question on SO, OP was using Element.hashCode() and it wasn't woring right.
The problem is that when jsoup generates the hashCode of an Element, the content text of the element will be ignored, and the hashCode is generated only based on the attributes, and the hashCode of the parent Element.

Using the following HTML:
<html>
    <head>
    </head>
    <body>
        <div style="blah">TODO: write content</div>
        <div style="blah">Nothing here</div>
        <p style="test">Empty</p>
        <p style="nothing">Empty</p>
    </body>
</html>

And the following code:
String html = //HTML posted above

Document doc = Jsoup.parse(html);

Elements elements = doc.select("[style]");
for (Element e : elements) {
   System.out.println(e.hashCode());
}

It gives:
-148184373
-148184373
-1050420242
2013043377

I believe the hashCode should be different for the first two Elements, since the content is text is different. Or is this intended behaviour?

Buggy code:
public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    if (!super.equals(o)) return false;

    Element element = (Element) o;

    return this == o;
}

Fixed code:
public boolean equals(Object o) {
    if (this == o) return true;
    if (o == null || getClass() != o.getClass()) return false;
    if (!super.equals(o)) return false;

    Element element = (Element) o;

    return tag.equals(element.tag);
}