Repository: jhy/jsoup
Pull request #2016 (2023-10-23 23:50:09)

Bump org.apache.maven.plugins:maven-failsafe-plugin from 3.1.2 to 3.2.1
Bumps [org.apache.maven.plugins:maven-failsafe-plugin](https://github.com/apache/maven-surefire) from 3.1.2 to 3.2.1.
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/apache/maven-surefire/commit/df9f657cb241c1f55ddde884249588080facffe3"><code>df9f657</code></a> [maven-release-plugin] prepare release surefire-3.2.1</li>
<li><a href="https://github.com/apache/maven-surefire/commit/0910b750c5d8cfa41e27a7c586d17012b7efcb1f"><code>0910b75</code></a> Fix formatting</li>
<li><a href="https://github.com/apache/maven-surefire/commit/97c84a1f9b83d1ac318d53d8ba8fba87b1c5462a"><code>97c84a1</code></a> [maven-release-plugin] prepare for next development iteration</li>
<li><a href="https://github.com/apache/maven-surefire/commit/ad0639f1f59aee4647641fd0681c416dd46ac484"><code>ad0639f</code></a> [maven-release-plugin] prepare release surefire-3.2.0</li>
<li><a href="https://github.com/apache/maven-surefire/commit/edfb96515553c548fddc58d2745d3e4c8e1367d3"><code>edfb965</code></a> [SUREFIRE-2196] maven-surefire-report-plugin:failsafe-report-only failed: Unm...</li>
<li><a href="https://github.com/apache/maven-surefire/commit/0a5dba8a3678aca1682a518e79f39502ef3935ea"><code>0a5dba8</code></a> [SUREFIRE-2188] Upgrade to Parent 40</li>
<li><a href="https://github.com/apache/maven-surefire/commit/5a0943805f7ef3da691bd4f39660346bfc7283d6"><code>5a09438</code></a> [SUREFIRE-2202] NullPointerException if super class meets specific condition</li>
<li><a href="https://github.com/apache/maven-surefire/commit/176937dea760e602d5fddd65b88d83974ed06a4b"><code>176937d</code></a> Simplify temp dir creation</li>
<li><a href="https://github.com/apache/maven-surefire/commit/9b79bdab7180aa00642e2d9dd85167e0fcaa662d"><code>9b79bda</code></a> [SUREFIRE-1124] Support forkNumber in environment variables</li>
<li><a href="https://github.com/apache/maven-surefire/commit/296696441dbabfdb8b25f58c30330974f9b04c8a"><code>2966964</code></a> Fix TestNG web site URL</li>
<li>Additional commits viewable in <a href="https://github.com/apache/maven-surefire/compare/surefire-3.1.2...surefire-3.2.1">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=org.apache.maven.plugins:maven-failsafe-plugin&package-manager=maven&previous-version=3.1.2&new-version=3.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>


Patch:
diff --git a/CHANGES b/CHANGES
index 9f05522c..197df4fa 100644
--- a/CHANGES
+++ b/CHANGES
@@ -1,6 +1,12 @@
 jsoup changelog
 
-Release 1.16.2 [PENDING]
+Release 1.17.1 [PENDING]
+  * Improvement: in the Elements list, added direct support for `#set(index, element)`, `#remove(index)`,
+    `#remove(object)`, `#clear()`, `#removeAll(collection)`, `#retainAll(collection)`, `#removeIf(filter)`,
+    `#replaceAll(operator)`. These methods update the original DOM, as well as the Elements list.
+    <https://github.com/jhy/jsoup/pull/2017>
+
+Release 1.16.2 [20-Oct-2023]
   * Improvement: optimized the performance of complex CSS selectors, by adding a cost-based query planner. Evaluators
     are sorted by their relative execution cost, and executed in order of lower to higher cost. This speeds the
     matching process by ensuring that simpler evaluations (such as a tag name match) are conducted prior to more
diff --git a/pom.xml b/pom.xml
index 268ba0b4..fd4cc156 100644
--- a/pom.xml
+++ b/pom.xml
@@ -5,7 +5,7 @@
 
   <groupId>org.jsoup</groupId>
   <artifactId>jsoup</artifactId>
-  <version>1.16.3-SNAPSHOT</version><!-- remember to update previous version below for japicmp -->
+  <version>1.17.1-SNAPSHOT</version><!-- remember to update previous version below for japicmp -->
   <description>jsoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors. jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers do.</description>
   <url>https://jsoup.org/</url>
   <inceptionYear>2009</inceptionYear>
@@ -24,7 +24,7 @@
     <url>https://github.com/jhy/jsoup</url>
     <connection>scm:git:https://github.com/jhy/jsoup.git</connection>
     <!-- <developerConnection>scm:git:git@github.com:jhy/jsoup.git</developerConnection> -->
-    <tag>jsoup-1.16.2</tag>
+    <tag>HEAD</tag>
   </scm>
   <organization>
     <name>Jonathan Hedley</name>
@@ -82,6 +82,8 @@
                 <ignore>java.util.function.Supplier</ignore>
                 <ignore>java.lang.ThreadLocal</ignore>
                 <ignore>java.io.UncheckedIOException</ignore>
+                <ignore>java.util.function.Predicate</ignore>
+                <ignore>java.util.function.UnaryOperator</ignore>
               </ignores>
               <!-- ^ Provided by https://developer.android.com/studio/write/java8-support#library-desugaring
                Possibly OK to remove androidscents; keep for now to validate other additions are supported. -->
@@ -176,7 +178,7 @@
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-surefire-plugin</artifactId>
-        <version>3.1.2</version>
+        <version>3.2.1</version>
         <configuration>
           <!-- smaller stack to find stack overflows -->
           <argLine>-Xss256k</argLine>
@@ -184,7 +186,7 @@
       </plugin>
       <plugin>
         <artifactId>maven-failsafe-plugin</artifactId>
-        <version>3.1.2</version>
+        <version>3.2.1</version>
         <executions>
           <execution>
             <goals>
@@ -202,14 +204,14 @@
         <!-- API version compat check - https://siom79.github.io/japicmp/ -->
         <groupId>com.github.siom79.japicmp</groupId>
         <artifactId>japicmp-maven-plugin</artifactId>
-        <version>0.18.1</version>
+        <version>0.18.2</version>
         <configuration>
           <!-- hard code previous version; can't detect when running stateless on build server -->
           <oldVersion>
             <dependency>
               <groupId>org.jsoup</groupId>
               <artifactId>jsoup</artifactId>
-              <version>1.16.1</version>
+              <version>1.16.2</version>
               <type>jar</type>
             </dependency>
           </oldVersion>
@@ -306,7 +308,7 @@
         <plugins>
           <plugin>
             <artifactId>maven-failsafe-plugin</artifactId>
-            <version>3.1.2</version>
+            <version>3.2.1</version>
             <executions>
               <execution>
                 <goals>
diff --git a/src/main/java/org/jsoup/select/Elements.java b/src/main/java/org/jsoup/select/Elements.java
index 31838e1f..a64f322b 100644
--- a/src/main/java/org/jsoup/select/Elements.java
+++ b/src/main/java/org/jsoup/select/Elements.java
@@ -14,14 +14,17 @@ import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Collection;
 import java.util.HashSet;
+import java.util.Iterator;
 import java.util.LinkedHashSet;
 import java.util.List;
+import java.util.function.Predicate;
+import java.util.function.UnaryOperator;
 
 /**
  A list of {@link Element}s, with methods that act on every element in the list.
- <p>
- To get an {@code Elements} object, use the {@link Element#select(String)} method.
- </p>
+ <p>To get an {@code Elements} object, use the {@link Element#select(String)} method.</p>
+ <p>Methods that {@link #set(int, Element) set}, {@link #remove(int) remove}, or {@link #replaceAll(UnaryOperator)
+ replace} Elements in the list will also act on the underlying {@link org.jsoup.nodes.Document DOM}.</p>
 
  @author Jonathan Hedley, jonathan@hedley.net */
 public class Elements extends ArrayList<Element> {
@@ -431,6 +434,7 @@ public class Elements extends ArrayList<Element> {
 
     /**
      * Remove each matched element from the DOM. This is similar to setting the outer HTML of each element to nothing.
+     * <p>The elements will still be retained in this list, in case further processing of them is desired.</p>
      * <p>
      * E.g. HTML: {@code <div><p>Hello</p> <p>there</p> <img /></div>}<br>
      * <code>doc.select("p").remove();</code><br>
@@ -440,6 +444,7 @@ public class Elements extends ArrayList<Element> {
      * @return this, for chaining
      * @see Element#empty()
      * @see #empty()
+     * @see #clear()
      */
     public Elements remove() {
         for (Element element : this) {
@@ -683,4 +688,121 @@ public class Elements extends ArrayList<Element> {
         return nodes;
     }
 
+    // list methods that update the DOM:
+
+    /**
+     Replace the Element at the specified index in this list, and in the DOM.
+     * @param index index of the element to replace
+     * @param element element to be stored at the specified position
+     * @return the old Element at this index
+     * @since 1.17.1
+     */
+    @Override public Element set(int index, Element element) {
+        Validate.notNull(element);
+        Element old = super.set(index, element);
+        old.replaceWith(element);
+        return old;
+    }
+
+    /**
+     Remove the Element at the specified index in this ist, and from the DOM.
+     * @param index the index of the element to be removed
+     * @return the old element at this index
+     * @since 1.17.1
+     */
+    @Override public Element remove(int index) {
+        Element old = super.remove(index);
+        old.remove();
+        return old;
+    }
+
+    /**
+     Remove the specified Element from this list, and from th DOM
+     * @param o element to be removed from this list, if present
+     * @return if this list contained the Element
+     * @since 1.17.1
+     */
+    @Override public boolean remove(Object o) {
+        int index = super.indexOf(o);
+        if (index == -1) {
+            return false;
+        } else {
+            remove(index);
+            return true;
+        }
+    }
+
+    /**
+     Removes all the elements from this list, and each of them from the DOM.
+     * @since 1.17.1
+     * @see #remove()
+     */
+    @Override public void clear() {
+        remove();
+        super.clear();
+    }
+
+    /**
+     Removes from this list, and from the DOM, each of the elements that are contained in the specified collection and
+     are in this list.
+     * @param c collection containing elements to be removed from this list
+     * @return {@code true} if elements were removed from this list
+     * @since 1.17.1
+     */
+    @Override public boolean removeAll(Collection<?> c) {
+        boolean anyRemoved = false;
+        for (Object o : c) {
+            anyRemoved |= this.remove(o);
+        }
+        return anyRemoved;
+    }
+
+    /**
+     Retain in this list, and in the DOM, only the elements that are in the specified collection and are in this list.
+     In other words, remove elements from this list and the DOM any item that is in this list but not in the specified
+     collection.
+     * @param c collection containing elements to be retained in this list
+     * @return {@code true} if elements were removed from this list
+     * @since 1.17.1
+     */
+    @Override public boolean retainAll(Collection<?> c) {
+        boolean anyRemoved = false;
+        for (Iterator<Element> it = this.iterator(); it.hasNext(); ) {
+            Element el = it.next();
+            if (!c.contains(el)) {
+                it.remove();
+                anyRemoved = true;
+            }
+        }
+        return anyRemoved;
+    }
+
+    /**
+     Remove from the list, and from the DOM, all elements in this list that mach the given filter.
+     * @param filter a predicate which returns {@code true} for elements to be removed
+     * @return {@code true} if elements were removed from this list
+     * @since 1.17.1
+     */
+    @Override public boolean removeIf(Predicate<? super Element> filter) {
+        boolean anyRemoved = false;
+        for (Iterator<Element> it = this.iterator(); it.hasNext(); ) {
+            Element el = it.next();
+            if (filter.test(el)) {
+                it.remove();
+                anyRemoved = true;
+            }
+        }
+        return anyRemoved;
+    }
+
+    /**
+     Replace each element in this list with the result of the operator, and update the DOM.
+     * @param operator the operator to apply to each element
+     * @since 1.17.1
+     */
+    @Override public void replaceAll(UnaryOperator<Element> operator) {
+        for (int i = 0; i < this.size(); i++) {
+            this.set(i, operator.apply(this.get(i)));
+        }
+    }
 }


Test patch:
diff --git a/src/test/java/org/jsoup/select/ElementsTest.java b/src/test/java/org/jsoup/select/ElementsTest.java
index d1895752..b5ea4ef3 100644
--- a/src/test/java/org/jsoup/select/ElementsTest.java
+++ b/src/test/java/org/jsoup/select/ElementsTest.java
@@ -11,9 +11,11 @@ import org.jsoup.nodes.Node;
 import org.jsoup.nodes.TextNode;
 import org.junit.jupiter.api.Test;
 
+import java.util.Iterator;
 import java.util.List;
 
 import static org.junit.jupiter.api.Assertions.*;
+import static org.junit.jupiter.api.Assertions.assertFalse;
 
 /**
  Tests for ElementList.
@@ -435,4 +437,166 @@ public class ElementsTest {
         assertEquals("http://example.com/bar", absAttrs.get(1));
         assertEquals("http://example.com", absAttrs.get(2));
     }
+
+    @Test public void setElementByIndex() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three");
+        Element newP = doc.createElement("p").text("New").attr("id", "new");
+
+        Elements ps = doc.select("p");
+        Element two = ps.get(1);
+        Element old = ps.set(1, newP);
+        assertSame(old, two);
+        assertSame(newP, ps.get(1)); // replaced in list
+        assertEquals("<p>One</p>\n<p id=\"new\">New</p>\n<p>Three</p>", doc.body().html()); // replaced in dom
+    }
+
+    @Test public void removeElementByIndex() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three");
+
+        Elements ps = doc.select("p");
+        Element two = ps.get(1);
+        assertTrue(ps.contains(two));
+        Element old = ps.remove(1);
+        assertSame(old, two);
+
+        assertEquals(2, ps.size()); // removed from list
+        assertFalse(ps.contains(old));
+        assertEquals("<p>One</p>\n<p>Three</p>", doc.body().html()); // removed from dom
+    }
+
+    @Test public void removeElementByObject() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three");
+
+        Elements ps = doc.select("p");
+        Element two = ps.get(1);
+        assertTrue(ps.contains(two));
+        boolean removed = ps.remove(two);
+        assertTrue(removed);
+
+        assertEquals(2, ps.size()); // removed from list
+        assertFalse(ps.contains(two));
+        assertEquals("<p>One</p>\n<p>Three</p>", doc.body().html()); // removed from dom
+    }
+
+    @Test public void removeElementObjectNoops() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three");
+        String origHtml = doc.html();
+        Element newP = doc.createElement("p").text("New");
+
+        Elements ps = doc.select("p");
+        int size = ps.size();
+        assertFalse(ps.remove(newP));
+        assertFalse(ps.remove(newP.childNodes()));
+        assertEquals(origHtml, doc.html());
+        assertEquals(size, ps.size());
+    }
+
+    @Test public void clear() {
+        Document doc = Jsoup.parse("<p>One</p><p>Two</p><div>Three</div>");
+        Elements ps = doc.select("p");
+        assertEquals(2, ps.size());
+        ps.clear();
+        assertEquals(0, ps.size());
+
+        assertEquals(0, doc.select("p").size());
+    }
+
+    @Test public void removeAll() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four</p><div>Div");
+        Elements ps = doc.select("p");
+        assertEquals(4, ps.size());
+        Elements midPs = doc.select("p:gt(0):lt(3)"); //Two and Three
+        assertEquals(2, midPs.size());
+
+        boolean removed = ps.removeAll(midPs);
+        assertEquals(2, ps.size());
+        assertTrue(removed);
+        assertEquals(2, midPs.size());
+
+        Elements divs = doc.select("div");
+        assertEquals(1, divs.size());
+        assertFalse(ps.removeAll(divs));
+        assertEquals(2, ps.size());
+
+        assertEquals("<p>One</p>\n<p>Four</p>\n<div>\n Div\n</div>", doc.body().html());
+    }
+
+    @Test public void retainAll() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four</p><div>Div");
+        Elements ps = doc.select("p");
+        assertEquals(4, ps.size());
+        Elements midPs = doc.select("p:gt(0):lt(3)"); //Two and Three
+        assertEquals(2, midPs.size());
+
+        boolean removed = ps.retainAll(midPs);
+        assertEquals(2, ps.size());
+        assertTrue(removed);
+        assertEquals(2, midPs.size());
+
+        assertEquals("<p>Two</p>\n<p>Three</p>\n<div>\n Div\n</div>", doc.body().html());
+
+        Elements psAgain = doc.select("p");
+        assertFalse(midPs.retainAll(psAgain));
+
+        assertEquals("<p>Two</p>\n<p>Three</p>\n<div>\n Div\n</div>", doc.body().html());
+    }
+
+    @Test public void iteratorRemovesFromDom() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four");
+        Elements ps = doc.select("p");
+
+        assertEquals(4, ps.size());
+        for (Iterator<Element> it = ps.iterator(); it.hasNext(); ) {
+            Element el = it.next();
+            if (el.text().contains("Two"))
+                it.remove();
+        }
+        assertEquals(3, ps.size());
+        assertEquals("<p>One</p>\n<p>Three</p>\n<p>Four</p>", doc.body().html());
+    }
+
+    @Test public void removeIf() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four");
+        Elements ps = doc.select("p");
+
+        assertEquals(4, ps.size());
+        boolean removed = ps.removeIf(el -> el.text().contains("Two"));
+        assertTrue(removed);
+        assertEquals(3, ps.size());
+        assertEquals("<p>One</p>\n<p>Three</p>\n<p>Four</p>", doc.body().html());
+
+        assertFalse(ps.removeIf(el -> el.text().contains("Five")));
+        assertEquals("<p>One</p>\n<p>Three</p>\n<p>Four</p>", doc.body().html());
+    }
+
+    @Test public void removeIfSupportsConcurrentRead() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four");
+        Elements ps = doc.select("p");
+        assertEquals(4, ps.size());
+
+        boolean removed = ps.removeIf(el -> ps.contains(el));
+        assertTrue(removed);
+        assertEquals(0, ps.size());
+        assertEquals("", doc.body().html());
+    }
+
+    @Test public void replaceAll() {
+        Document doc = Jsoup.parse("<p>One<p>Two<p>Three<p>Four");
+        Elements ps = doc.select("p");
+        assertEquals(4, ps.size());
+
+        ps.replaceAll(el -> {
+            Element div = doc.createElement("div");
+            div.text(el.text());
+            return div;
+        });
+
+        // Check Elements
+        for (Element p : ps) {
+            assertEquals("div", p.tagName());
+        }
+
+        // check dom
+        assertEquals("<div> One</div><div> Two</div><div> Three</div><div> Four</div>", TextUtil.normalizeSpaces(doc.body().html()));
+    }
 }