Repository: jhy/jsoup
Pull request #1924 (2023-03-27 23:56:44)

Bump animal-sniffer-maven-plugin from 1.22 to 1.23
Bumps [animal-sniffer-maven-plugin](https://github.com/mojohaus/animal-sniffer) from 1.22 to 1.23.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/mojohaus/animal-sniffer/releases">animal-sniffer-maven-plugin's releases</a>.</em></p>
<blockquote>
<h2>1.23</h2>
<!-- raw HTML omitted -->
<h2>🚀 New features and improvements</h2>
<ul>
<li>Get rid of maven-compat use Resolver API (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/239">#239</a>) <a href="https://github.com/slawekjaranowski"><code>@​slawekjaranowski</code></a></li>
<li>Update plugin, require Maven 3.2.5 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/223">#223</a>) <a href="https://github.com/cstamas"><code>@​cstamas</code></a></li>
<li>Improve field handling (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/212">#212</a>) <a href="https://github.com/Marcono1234"><code>@​Marcono1234</code></a></li>
</ul>
<h2>📦 Dependency updates</h2>
<ul>
<li>Bump parent from 70 to 74 and poms cleanup (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/241">#241</a>) <a href="https://github.com/slawekjaranowski"><code>@​slawekjaranowski</code></a></li>
<li>Bump maven-surefire-plugin from 2.22.2 to 3.0.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/238">#238</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump plexus-utils from 3.5.0 to 3.5.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/236">#236</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-plugin from 3.7.1 to 3.8.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/234">#234</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-annotations from 3.7.1 to 3.8.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/235">#235</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump enforcer-api from 3.1.0 to 3.2.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/232">#232</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-enforcer-plugin from 3.1.0 to 3.2.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/231">#231</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-plugin from 3.7.0 to 3.7.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/228">#228</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump ant from 1.10.12 to 1.10.13 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/226">#226</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-annotations from 3.7.0 to 3.7.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/229">#229</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-checkstyle-plugin from 3.2.0 to 3.2.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/227">#227</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-annotations from 3.6.4 to 3.7.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/221">#221</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-plugin-plugin from 3.6.4 to 3.7.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/220">#220</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump asm from 9.3 to 9.4 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/217">#217</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-shade-plugin from 3.4.0 to 3.4.1 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/219">#219</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump plexus-utils from 3.4.2 to 3.5.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/218">#218</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-common-artifact-filters from 3.3.1 to 3.3.2 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/216">#216</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-shade-plugin from 3.3.0 to 3.4.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/215">#215</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
<li>Bump maven-checkstyle-plugin from 3.1.2 to 3.2.0 (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/213">#213</a>) <a href="https://github.com/dependabot"><code>@​dependabot</code></a></li>
</ul>
<h2>👻 Maintenance</h2>
<ul>
<li>Refresh documentation site (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/242">#242</a>) <a href="https://github.com/slawekjaranowski"><code>@​slawekjaranowski</code></a></li>
<li>Prepare to run on Java 19+ (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/237">#237</a>) <a href="https://github.com/slachiewicz"><code>@​slachiewicz</code></a></li>
<li>Update usage.apt.vm (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/222">#222</a>) <a href="https://github.com/hbeni"><code>@​hbeni</code></a></li>
<li>Use HTTPS links in README and replace Travis badge (<a href="https://redirect.github.com/mojohaus/animal-sniffer/pull/211">#211</a>) <a href="https://github.com/Marcono1234"><code>@​Marcono1234</code></a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/bae24234f6611b434d250ef47bbba101f3f2781b"><code>bae2423</code></a> [maven-release-plugin] prepare release 1.23</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/90f22e4a9d7588a2fe13d7e619858448f99a50bc"><code>90f22e4</code></a> Refresh documentation site</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/49fdd5d8e3f7c7d7f4e1543cf8e03a439be0b32d"><code>49fdd5d</code></a> Bump parent from 70 to 74 and poms cleanup</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/fa0f098d3e3ef1ea67b980a0163ce9943c63261b"><code>fa0f098</code></a> Get rid of maven-compat use Resolver API</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/4f547b0bf646cf5c6ad024f5fe04c0b6dc4dfc96"><code>4f547b0</code></a> (ci) Use latest releases of Maven 3.9.1 and 3.8.8</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/ac9296bf30e9b5b150fa47b19e39446299caca4e"><code>ac9296b</code></a> Bump maven-surefire-plugin from 2.22.2 to 3.0.0</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/f343158c4af67cb1af514fdba862a30edf4e27b4"><code>f343158</code></a> Prepare to run on Java 19+</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/797ccf730cd4308644bc6076dbf246dcfa064fc8"><code>797ccf7</code></a> Bump plexus-utils from 3.5.0 to 3.5.1</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/2ce6d4367e54a7a01b40ebca04b3613d41254134"><code>2ce6d43</code></a> Bump maven-plugin-plugin from 3.7.1 to 3.8.1</li>
<li><a href="https://github.com/mojohaus/animal-sniffer/commit/9452d08ec7a66290a137bb25cb04ac52c49b88e1"><code>9452d08</code></a> Bump maven-plugin-annotations from 3.7.1 to 3.8.1</li>
<li>Additional commits viewable in <a href="https://github.com/mojohaus/animal-sniffer/compare/animal-sniffer-parent-1.22...1.23">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.codehaus.mojo:animal-sniffer-maven-plugin&package-manager=maven&previous-version=1.22&new-version=1.23)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Patch:
diff --git a/CHANGES b/CHANGES
index b4cc2723..b6770a21 100644
--- a/CHANGES
+++ b/CHANGES
@@ -24,6 +24,9 @@ Release 1.16.1 [PENDING]
     again, causing errors when fetched.
     <https://github.com/jhy/jsoup/issues/1902>
 
+  * Bugfix: when tracking input source positions, text in tables that was fostered had invalid positions.
+    <https://github.com/jhy/jsoup/issues/1927>
+
   * Bugfix: If the Document.OutputSettings class was initialized, and then Entities.escape(String) called, an NPE may be
     thrown due to a class loading circular dependency.
     <https://github.com/jhy/jsoup/issues/1910>
diff --git a/pom.xml b/pom.xml
index 62bf6d6c..a36a6feb 100644
--- a/pom.xml
+++ b/pom.xml
@@ -58,7 +58,7 @@
         <!-- Ensure Java 8 and Android 10 API compatibility -->
         <groupId>org.codehaus.mojo</groupId>
         <artifactId>animal-sniffer-maven-plugin</artifactId>
-        <version>1.22</version>
+        <version>1.23</version>
         <executions>
           <execution>
             <id>animal-sniffer</id>
@@ -164,7 +164,7 @@
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-resources-plugin</artifactId>
-        <version>3.3.0</version>
+        <version>3.3.1</version>
       </plugin>
       <plugin>
         <artifactId>maven-release-plugin</artifactId>
diff --git a/src/main/java/org/jsoup/parser/HtmlTreeBuilder.java b/src/main/java/org/jsoup/parser/HtmlTreeBuilder.java
index 9de525b6..06e9c749 100644
--- a/src/main/java/org/jsoup/parser/HtmlTreeBuilder.java
+++ b/src/main/java/org/jsoup/parser/HtmlTreeBuilder.java
@@ -53,7 +53,7 @@ public class HtmlTreeBuilder extends TreeBuilder {
     private @Nullable Element contextElement; // fragment parse context -- could be null even if fragment parsing
     private ArrayList<Element> formattingElements; // active (open) formatting elements
     private ArrayList<HtmlTreeBuilderState> tmplInsertMode; // stack of Template Insertion modes
-    private List<String> pendingTableCharacters; // chars in table to be shifted out
+    private List<Token.Character> pendingTableCharacters; // chars in table to be shifted out
     private Token.EndTag emptyEnd; // reused empty end tag
 
     private boolean framesetOk; // if ok to go into frameset
@@ -676,14 +676,20 @@ public class HtmlTreeBuilder extends TreeBuilder {
         this.formElement = formElement;
     }
 
-    void newPendingTableCharacters() {
+    void resetPendingTableCharacters() {
         pendingTableCharacters = new ArrayList<>();
     }
 
-    List<String> getPendingTableCharacters() {
+    List<Token.Character> getPendingTableCharacters() {
         return pendingTableCharacters;
     }
 
+    void addPendingTableCharacters(Token.Character c) {
+        // make a clone of the token to maintain its state (as Tokens are otherwise reset)
+        Token.Character clone = c.clone();
+        pendingTableCharacters.add(clone);
+    }
+
     /**
      13.2.6.3 Closing elements that have implied end tags
      When the steps below require the UA to generate implied end tags, then, while the current node is a dd element, a dt element, an li element, an optgroup element, an option element, a p element, an rb element, an rp element, an rt element, or an rtc element, the UA must pop the current node off the stack of open elements.
diff --git a/src/main/java/org/jsoup/parser/HtmlTreeBuilderState.java b/src/main/java/org/jsoup/parser/HtmlTreeBuilderState.java
index 99edf8ce..3c5352ec 100644
--- a/src/main/java/org/jsoup/parser/HtmlTreeBuilderState.java
+++ b/src/main/java/org/jsoup/parser/HtmlTreeBuilderState.java
@@ -6,7 +6,6 @@ import org.jsoup.nodes.Attributes;
 import org.jsoup.nodes.Document;
 import org.jsoup.nodes.DocumentType;
 import org.jsoup.nodes.Element;
-import org.jsoup.nodes.Node;
 
 import java.util.ArrayList;
 
@@ -995,7 +994,7 @@ enum HtmlTreeBuilderState {
     InTable {
         boolean process(Token t, HtmlTreeBuilder tb) {
             if (t.isCharacter() && inSorted(tb.currentElement().normalName(), InTableFoster)) {
-                tb.newPendingTableCharacters();
+                tb.resetPendingTableCharacters();
                 tb.markInsertionMode();
                 tb.transition(InTableText);
                 return tb.process(t);
@@ -1106,25 +1105,25 @@ enum HtmlTreeBuilderState {
                     tb.error(this);
                     return false;
                 } else {
-                    tb.getPendingTableCharacters().add(c.getData());
+                    tb.addPendingTableCharacters(c);
                 }
-            } else {// todo - don't really like the way these table character data lists are built
+            } else {
                 if (tb.getPendingTableCharacters().size() > 0) {
-                    for (String character : tb.getPendingTableCharacters()) {
-                        if (!isWhitespace(character)) {
+                    for (Token.Character c : tb.getPendingTableCharacters()) {
+                        if (!isWhitespace(c)) {
                             // InTable anything else section:
                             tb.error(this);
                             if (inSorted(tb.currentElement().normalName(), InTableFoster)) {
                                 tb.setFosterInserts(true);
-                                tb.process(new Token.Character().data(character), InBody);
+                                tb.process(c, InBody);
                                 tb.setFosterInserts(false);
                             } else {
-                                tb.process(new Token.Character().data(character), InBody);
+                                tb.process(c, InBody);
                             }
                         } else
-                            tb.insert(new Token.Character().data(character));
+                            tb.insert(c);
                     }
-                    tb.newPendingTableCharacters();
+                    tb.resetPendingTableCharacters();
                 }
                 tb.transition(tb.originalState());
                 return tb.process(t);
@@ -1759,10 +1758,6 @@ enum HtmlTreeBuilderState {
         return false;
     }
 
-    private static boolean isWhitespace(String data) {
-        return StringUtil.isBlank(data);
-    }
-
     private static void handleRcData(Token.StartTag startTag, HtmlTreeBuilder tb) {
         tb.tokeniser.transition(TokeniserState.Rcdata);
         tb.markInsertionMode();
diff --git a/src/main/java/org/jsoup/parser/Token.java b/src/main/java/org/jsoup/parser/Token.java
index 819b8aef..b0fc0af8 100644
--- a/src/main/java/org/jsoup/parser/Token.java
+++ b/src/main/java/org/jsoup/parser/Token.java
@@ -382,7 +382,7 @@ abstract class Token {
         }
     }
 
-    static class Character extends Token {
+    static class Character extends Token implements Cloneable {
         private String data;
 
         Character() {
@@ -410,6 +410,14 @@ abstract class Token {
         public String toString() {
             return getData();
         }
+
+        @Override protected Token.Character clone() {
+            try {
+                return (Token.Character) super.clone();
+            } catch (CloneNotSupportedException e) {
+                throw new RuntimeException(e);
+            }
+        }
     }
 
     final static class CData extends Character {


Test patch:
diff --git a/src/test/java/org/jsoup/nodes/PositionTest.java b/src/test/java/org/jsoup/nodes/PositionTest.java
index 813ed9b3..78df6fbb 100644
--- a/src/test/java/org/jsoup/nodes/PositionTest.java
+++ b/src/test/java/org/jsoup/nodes/PositionTest.java
@@ -3,9 +3,12 @@ package org.jsoup.nodes;
 import org.jsoup.Jsoup;
 import org.jsoup.integration.servlets.FileServlet;
 import org.jsoup.parser.Parser;
+import org.jsoup.select.NodeTraversor;
 import org.junit.jupiter.api.Test;
 
 import java.io.IOException;
+import java.util.ArrayList;
+import java.util.List;
 
 import static org.junit.jupiter.api.Assertions.*;
 
@@ -169,4 +172,23 @@ class PositionTest {
         assertEquals("17,5:779-17,12:786", item.endSourceRange().toString());
     }
 
+    @Test void tracksTableMovedText() {
+        String html = "<table>foo<tr>bar<td>baz</td>qux</tr>coo</table>";
+        Document doc = Jsoup.parse(html, TrackingParser);
+
+        List<TextNode> textNodes = new ArrayList<>();
+        NodeTraversor.traverse((Node node, int depth) -> {
+            if (node instanceof TextNode) {
+                textNodes.add((TextNode) node);
+            }
+        }, doc);
+
+        assertEquals(5, textNodes.size());
+        assertEquals("1,8:7-1,11:10", textNodes.get(0).sourceRange().toString());
+        assertEquals("1,15:14-1,18:17", textNodes.get(1).sourceRange().toString());
+        assertEquals("1,22:21-1,25:24", textNodes.get(2).sourceRange().toString());
+        assertEquals("1,30:29-1,33:32", textNodes.get(3).sourceRange().toString());
+        assertEquals("1,38:37-1,41:40", textNodes.get(4).sourceRange().toString());
+    }
+
 }
\ No newline at end of file