Repository: elastic/logstash
Pull request #17203: Pluginmanager clean after mutate
Author: login: yaauie
Labels: name: backport-9.0
description: Automated backport to the 9.0 branch with mergify
name: backport-8.x
description: Automated backport to the 8.x branch with mergify
Created: 2025-02-28T22:28:52Z | Merged: 2025-03-04T18:59:35Z

## Release notes

 - FIX: The plugin manager now correctly cleans up plugins and gem dependencies that are no longer needed after invoking `install`, `remove`, or `update`.
 - FIX: The plugin manager's `update` command no longer fails to upgrade plugins that were initially installed with `--version` flag

## What does this PR do?

 - Takes the previously-reviewed guts of https://github.com/elastic/logstash/pull/16998 and attaches the `clean` operation to the successful completions of the `update`, `remove`, and `install` commands _instead of_ as its own stand-alone command.
 - Fixes https://github.com/elastic/logstash/issues/17196, in which the `update` command fails to update plugins that were initially installed with a `--version` flag

## Why is it important/What is the impact to the user?

  - For users who update, install, or remove plugins, cleaning out deactivated dependencies can reduce the disk footprint of the logstash installation
  - For building Logstash artifacts, this allows us to build a true subset of the dependency graph after removing plugins that are not needed in a minimalized artifact

## Checklist

<!-- Mandatory
Add a checklist of things that are required to be reviewed in order to have the PR approved

List here all the items you have verified BEFORE sending this PR. Please DO NOT remove any item, striking through those that do not apply. (Just in case, strikethrough uses two tildes. ~~Scratch this.~~)
-->

- [x] My code follows the style guidelines of this project
- [x] I have commented my code, particularly in hard-to-understand areas
- ~~[ ] I have made corresponding changes to the documentation~~
- [x] I have made corresponding change to the default configuration files (and/or docker env variables)
- [x] I have added tests that prove my fix is effective or that my feature works

## Author's Checklist

<!-- Recommended
Add a checklist of things that are required to be reviewed in order to have the PR approved
-->
- [ ]

## How to test this PR locally

Likely the easiest is to install, remove, or update plugins, observe the output, and check how the `vendor/bundle` was changed:

1. Installing a plugin should remove overwritten plugins and orphaned dependencies
   ~~~
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt
   bin/logstash-plugin install --version 4.1.2 logstash-output-email
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt
   diff -u vendor_snapshot.txt vendor_compare.txt
   ~~~
2. Updating a plugin should remove the previous version (and its shared dependencies)
   ~~~
   bin/logstash-plugin install --version 4.1.2 logstash-output-email
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt
   bin/logstash-plugin update logstash-output-email
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt
   diff -u vendor_snapshot.txt vendor_compare.txt
   ~~~
3. Removing a plugin should remove the plugin gem _and_ its unshared dependencies:
   ~~~
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_snapshot.txt
   bin/logstash-plugin remove logstash-integration-aws
   find vendor/bundle/jruby/*/* -depth 1 | sort > vendor_compare.txt
   diff -u vendor_snapshot.txt vendor_compare.txt
   ~~~

## Related issues

 - Resolves: https://github.com/elastic/logstash/issues/17196
 - Supersedes: https://github.com/elastic/logstash/pull/16998




Diff:
diff --git a/lib/pluginmanager/command.rb b/lib/pluginmanager/command.rb
index 8fd700db564..5603dc383a1 100644
--- a/lib/pluginmanager/command.rb
+++ b/lib/pluginmanager/command.rb
@@ -47,6 +47,31 @@ def remove_unused_locally_installed_gems!
     end
   end
 
+  def remove_orphan_dependencies!
+    locked_gem_names = ::Bundler::LockfileParser.new(File.read(LogStash::Environment::LOCKFILE)).specs.map(&:full_name).to_set
+    orphan_gem_specs = ::Gem::Specification.each
+                                           .reject(&:stubbed?) # skipped stubbed (uninstalled) gems
+                                           .reject(&:default_gem?) # don't touch jruby-included default gems
+                                           .reject{ |spec| locked_gem_names.include?(spec.full_name) }
+                                           .sort
+
+    inactive_plugins, orphaned_dependencies = orphan_gem_specs.partition { |spec| LogStash::PluginManager.logstash_plugin_gem_spec?(spec) }
+
+    # uninstall plugins first, to limit damage should one fail to uninstall
+    inactive_plugins.each { |plugin| uninstall_gem!("inactive plugin", plugin) }
+    orphaned_dependencies.each { |dep| uninstall_gem!("orphaned dependency", dep) }
+  end
+
+  def uninstall_gem!(desc, spec)
+    require "rubygems/uninstaller"
+    Gem::DefaultUserInteraction.use_ui(debug? ? Gem::DefaultUserInteraction.ui : Gem::SilentUI.new) do
+      Gem::Uninstaller.new(spec.name, version: spec.version, force: true, executables: true).uninstall
+    end
+    puts "cleaned #{desc} #{spec.name} (#{spec.version})"
+  rescue Gem::InstallError => e
+    report_exception("Failed to uninstall `#{spec.full_name}`", e)
+  end
+
   def relative_path(path)
     require "pathname"
     ::Pathname.new(path).relative_path_from(::Pathname.new(LogStash::Environment::LOGSTASH_HOME)).to_s
diff --git a/lib/pluginmanager/gemfile.rb b/lib/pluginmanager/gemfile.rb
index 6e658b762d8..7e295a9fcf1 100644
--- a/lib/pluginmanager/gemfile.rb
+++ b/lib/pluginmanager/gemfile.rb
@@ -133,8 +133,8 @@ def add_gem(_gem)
     # update existing or add new
     def update_gem(_gem)
       if old = find_gem(_gem.name)
-        # always overwrite requirements if specified
-        old.requirements = _gem.requirements unless no_constrains?(_gem.requirements)
+        # always overwrite requirements
+        old.requirements = _gem.requirements
         # but merge options
         old.options = old.options.merge(_gem.options)
       else
diff --git a/lib/pluginmanager/install.rb b/lib/pluginmanager/install.rb
index 085b6430214..e1a3513987d 100644
--- a/lib/pluginmanager/install.rb
+++ b/lib/pluginmanager/install.rb
@@ -79,6 +79,7 @@ def execute
     install_gems_list!(gems)
     remove_unused_locally_installed_gems!
     remove_unused_integration_overlaps!
+    remove_orphan_dependencies!
   end
 
   private
diff --git a/lib/pluginmanager/remove.rb b/lib/pluginmanager/remove.rb
index 36a8592f326..48d832abcd3 100644
--- a/lib/pluginmanager/remove.rb
+++ b/lib/pluginmanager/remove.rb
@@ -67,6 +67,7 @@ def execute
     exit(1) unless ::Bundler::LogstashUninstall.uninstall!(plugin_list)
     LogStash::Bundler.genericize_platform
     remove_unused_locally_installed_gems!
+    remove_orphan_dependencies!
   rescue => exception
     report_exception("Operation aborted, cannot remove plugin", exception)
   end
diff --git a/lib/pluginmanager/update.rb b/lib/pluginmanager/update.rb
index 76130bfa584..91306bc1e85 100644
--- a/lib/pluginmanager/update.rb
+++ b/lib/pluginmanager/update.rb
@@ -95,10 +95,8 @@ def update_gems!
       output << LogStash::Bundler.genericize_platform unless output.nil?
     end
 
-    # We currently dont removed unused gems from the logstash installation
-    # see: https://github.com/elastic/logstash/issues/6339
-    # output = LogStash::Bundler.invoke!(:clean => true)
     display_updated_plugins(previous_gem_specs_map)
+    remove_orphan_dependencies!
   rescue => exception
     gemfile.restore!
     report_exception("Updated Aborted", exception)
diff --git a/qa/integration/fixtures/update_spec.yml b/qa/integration/fixtures/update_spec.yml
new file mode 100644
index 00000000000..cbfc784af81
--- /dev/null
+++ b/qa/integration/fixtures/update_spec.yml
@@ -0,0 +1,3 @@
+---
+services:
+  - logstash
diff --git a/qa/integration/services/logstash_service.rb b/qa/integration/services/logstash_service.rb
index b4881807a8a..e3e1985644a 100644
--- a/qa/integration/services/logstash_service.rb
+++ b/qa/integration/services/logstash_service.rb
@@ -306,7 +306,7 @@ def run_cmd(cmd_args, change_dir = true, environment = {})
     if ENV.key?("BUILD_JAVA_HOME") && !process.environment.key?("LS_JAVA_HOME")
       process.environment["LS_JAVA_HOME"] = ENV["BUILD_JAVA_HOME"]
     end
-    process.io.stdout = process.io.stderr = out
+    process.io.stdout = process.io.stderr = SynchronizedDelegate.new(out)
 
     Bundler.with_unbundled_env do
       if change_dir
@@ -327,6 +327,31 @@ def run(*args)
     run_cmd [@logstash_bin, *args]
   end
 
+  ##
+  # A `SynchronizedDelegate` wraps any object and ensures that exactly one
+  # calling thread is invoking methods on it at a time. This is useful for our
+  # clumsy setting of process io STDOUT and STDERR to the same IO object, which
+  # can cause interleaved writes.
+  class SynchronizedDelegate
+    def initialize(obj)
+      require "monitor"
+      @mon = Monitor.new
+      @obj = obj
+    end
+
+    def respond_to_missing?(method_name, include_private = false)
+      @obj.respond_to?(method_name, include_private) || super
+    end
+
+    def method_missing(method_name, *args, &block)
+      return super unless @obj.respond_to?(method_name)
+
+      @mon.synchronize do
+        @obj.public_send(method_name, *args, &block)
+      end
+    end
+  end
+
   class PluginCli
 
     LOGSTASH_PLUGIN = File.join("bin", "logstash-plugin")
@@ -360,9 +385,26 @@ def list(*plugins, verbose: false)
       run(command)
     end
 
-    def install(plugin_name, *additional_plugins)
-      plugin_list = ([plugin_name]+additional_plugins).flatten
-      run("install #{Shellwords.shelljoin(plugin_list)}")
+    def install(plugin_name, *additional_plugins, version: nil, verify: true, preserve: false, local: false)
+      args = []
+      args << "--no-verify" unless verify
+      args << "--preserve" if preserve
+      args << "--local" if local
+      args << "--version" << version unless version.nil?
+      args.concat(([plugin_name]+additional_plugins).flatten)
+
+      run("install #{Shellwords.shelljoin(args)}")
+    end
+
+    def update(*plugin_list, level: nil, local: nil, verify: nil, conservative: nil)
+      args = []
+      args << (verify ? "--verify" : "--no-verify") unless verify.nil?
+      args << "--level" << "#{level}" unless level.nil?
+      args << "--local" if local
+      args << (conservative ? "--conservative" : "--no-conservative") unless conservative.nil?
+      args.concat(plugin_list)
+
+      run("update #{Shellwords.shelljoin(args)}")
     end
 
     def run(command)
diff --git a/qa/integration/specs/cli/install_spec.rb b/qa/integration/specs/cli/install_spec.rb
index 89a97973c6e..c3b55f926e3 100644
--- a/qa/integration/specs/cli/install_spec.rb
+++ b/qa/integration/specs/cli/install_spec.rb
@@ -19,6 +19,7 @@
 require_relative "../../framework/settings"
 require_relative "../../services/logstash_service"
 require_relative "../../framework/helpers"
+require_relative "pluginmanager_spec_helper"
 require "logstash/devutils/rspec/spec_helper"
 require "stud/temporary"
 require "fileutils"
@@ -29,16 +30,21 @@ def gem_in_lock_file?(pattern, lock_file)
   content.match(pattern)
 end
 
+def plugin_filename_re(name, version)
+  %Q(\b#{Regexp.escape name}-#{Regexp.escape version}(-java)?\b)
+end
+
 # Bundler can mess up installation successful output: https://github.com/elastic/logstash/issues/15801
 INSTALL_SUCCESS_RE = /IB?nstall successful/
 INSTALLATION_SUCCESS_RE = /IB?nstallation successful/
 
+INSTALLATION_ABORTED_RE = /Installation aborted/
+
 describe "CLI > logstash-plugin install" do
-  before(:all) do
+  before(:each) do
     @fixture = Fixture.new(__FILE__)
     @logstash = @fixture.get_service("logstash")
     @logstash_plugin = @logstash.plugin_cli
-    @pack_directory =  File.expand_path(File.join(File.dirname(__FILE__), "..", "..", "fixtures", "logstash-dummy-pack"))
   end
 
   shared_examples "install from a pack" do
@@ -46,6 +52,10 @@ def gem_in_lock_file?(pattern, lock_file)
     let(:install_command) { "bin/logstash-plugin install" }
     let(:change_dir) { true }
 
+    before(:all) do
+      @pack_directory =  File.expand_path(File.join(File.dirname(__FILE__), "..", "..", "fixtures", "logstash-dummy-pack"))
+    end
+
     # When you are on anything by linux we won't disable the internet with seccomp
     if RbConfig::CONFIG["host_os"] == "linux"
       context "without internet connection (linux seccomp wrapper)" do
@@ -152,4 +162,92 @@ def gem_in_lock_file?(pattern, lock_file)
       end
     end
   end
+
+  context "rubygems hosted plugin" do
+    include_context "pluginmanager validation helpers"
+    shared_examples("overwriting existing") do
+      before(:each) do
+        aggregate_failures("precheck") do
+          expect("#{plugin_name}-#{existing_plugin_version}").to_not be_installed_gem
+          expect("#{plugin_name}-#{specified_plugin_version}").to_not be_installed_gem
+        end
+        aggregate_failures("setup") do
+          execute = @logstash_plugin.install(plugin_name, version: existing_plugin_version)
+
+          expect(execute.stderr_and_stdout).to match(INSTALLATION_SUCCESS_RE)
+          expect(execute.exit_code).to eq(0)
+
+          expect("#{plugin_name}-#{existing_plugin_version}").to be_installed_gem
+          expect("#{plugin_name}-#{specified_plugin_version}").to_not be_installed_gem
+        end
+      end
+      it "installs the specified version and removes the pre-existing one" do
+        execute = @logstash_plugin.install(plugin_name, version: specified_plugin_version)
+
+        aggregate_failures("command execution") do
+          expect(execute.stderr_and_stdout).to match(INSTALLATION_SUCCESS_RE)
+          expect(execute.exit_code).to eq(0)
+        end
+
+        installed = @logstash_plugin.list(plugin_name, verbose: true)
+        expect(installed.stderr_and_stdout).to match(/#{Regexp.escape plugin_name} [(]#{Regexp.escape(specified_plugin_version)}[)]/)
+
+        expect("#{plugin_name}-#{existing_plugin_version}").to_not be_installed_gem
+        expect("#{plugin_name}-#{specified_plugin_version}").to be_installed_gem
+      end
+    end
+
+    context "when installing over an older version" do
+      let(:plugin_name) { "logstash-filter-qatest" }
+      let(:existing_plugin_version) { "0.1.0" }
+      let(:specified_plugin_version) { "0.1.1" }
+
+      include_examples "overwriting existing"
+    end
+
+    context "when installing over a newer version" do
+      let(:plugin_name) { "logstash-filter-qatest" }
+      let(:existing_plugin_version) { "0.1.0" }
+      let(:specified_plugin_version) { "0.1.1" }
+
+      include_examples "overwriting existing"
+    end
+
+    context "installing plugin that isn't present" do
+      it "installs the plugin" do
+        aggregate_failures("prevalidation") do
+          expect("logstash-filter-qatest").to_not be_installed_gem
+        end
+
+        execute = @logstash_plugin.install("logstash-filter-qatest")
+
+        expect(execute.stderr_and_stdout).to match(INSTALLATION_SUCCESS_RE)
+        expect(execute.exit_code).to eq(0)
+
+        installed = @logstash_plugin.list("logstash-filter-qatest")
+        expect(installed.stderr_and_stdout).to match(/logstash-filter-qatest/)
+        expect(installed.exit_code).to eq(0)
+
+        expect(gem_in_lock_file?(/logstash-filter-qatest/, @logstash.lock_file)).to be_truthy
+
+        expect("logstash-filter-qatest").to be_installed_gem
+      end
+    end
+    context "installing plugin that doesn't exist on rubygems" do
+      it "doesn't install anything" do
+        execute = @logstash_plugin.install("logstash-filter-404-no-exist")
+
+        expect(execute.stderr_and_stdout).to match(INSTALLATION_ABORTED_RE)
+        expect(execute.exit_code).to eq(1)
+      end
+    end
+    context "installing gem that isn't a plugin" do
+      it "doesn't install anything" do
+        execute = @logstash_plugin.install("dummy_gem")
+
+        expect(execute.stderr_and_stdout).to match(INSTALLATION_ABORTED_RE)
+        expect(execute.exit_code).to eq(1)
+      end
+    end
+  end
 end
diff --git a/qa/integration/specs/cli/pluginmanager_spec_helper.rb b/qa/integration/specs/cli/pluginmanager_spec_helper.rb
new file mode 100644
index 00000000000..6270f7933c1
--- /dev/null
+++ b/qa/integration/specs/cli/pluginmanager_spec_helper.rb
@@ -0,0 +1,59 @@
+require 'pathname'
+
+shared_context "pluginmanager validation helpers" do
+
+  matcher :be_installed_gem do
+    match do |actual|
+      common(actual)
+      @gemspec_present && @gem_installed
+    end
+
+    match_when_negated do |actual|
+      common(actual)
+      !@gemspec_present && !@gem_installed
+    end
+
+    define_method :common do |actual|
+      version_suffix = /-[0-9.]+(-java)?$/
+      filename_matcher = actual.match?(version_suffix) ? actual : /^#{Regexp.escape(actual)}#{version_suffix}/
+
+      @gems = (logstash_gemdir / "gems").glob("*-*")
+      @gemspecs = (logstash_gemdir / "specifications").glob("*-*.gemspec")
+
+      @gem_installed = @gems.find { |gem| gem.basename.to_s.match?(filename_matcher) }
+      @gemspec_present = @gemspecs.find { |gemspec| gemspec.basename(".gemspec").to_s.match?(filename_matcher) }
+    end
+
+    failure_message do |actual|
+      reasons = []
+      reasons << "the gem dir could not be found (#{@gems})" unless @gem_installed
+      reasons << "the gemspec could not be found (#{@gemspecs})" unless @gemspec_present
+
+      "expected that #{actual} would be installed, but #{reasons.join(' and ')}"
+    end
+    failure_message_when_negated do |actual|
+      reasons = []
+      reasons << "the gem dir is present (#{@gem_installed})" if @gem_installed
+      reasons << "the gemspec is present (#{@gemspec_present})" if @gemspec_present
+
+      "expected that #{actual} would not be installed, but #{reasons.join(' and ')}"
+    end
+  end
+
+  def logstash_home
+    return super() if defined?(super)
+    return @logstash.logstash_home if @logstash
+    fail("no @logstash, so we can't get logstash_home")
+  end
+
+  def logstash_gemdir
+    pathname_base = (Pathname.new(logstash_home) / "vendor" / "bundle" / "jruby")
+    candidate_dirs = pathname_base.glob("[0-9]*")
+    case candidate_dirs.size
+    when 0 then fail("no version dir found in #{pathname_base}")
+    when 1 then candidate_dirs.first
+    else
+      fail("multiple version dirs found in #{pathname_base} (#{candidate_dirs.map(&:basename)}")
+    end
+  end
+end
\ No newline at end of file
diff --git a/qa/integration/specs/cli/remove_spec.rb b/qa/integration/specs/cli/remove_spec.rb
index b3d914f6565..bf168aafc4f 100644
--- a/qa/integration/specs/cli/remove_spec.rb
+++ b/qa/integration/specs/cli/remove_spec.rb
@@ -19,12 +19,17 @@
 require_relative '../../framework/settings'
 require_relative '../../services/logstash_service'
 require_relative '../../framework/helpers'
+require_relative "pluginmanager_spec_helper"
 require "logstash/devutils/rspec/spec_helper"
 
 describe "CLI > logstash-plugin remove" do
+
+  include_context "pluginmanager validation helpers"
+
   before(:each) do
     @fixture = Fixture.new(__FILE__)
-    @logstash_plugin = @fixture.get_service("logstash").plugin_cli
+    @logstash = @fixture.get_service("logstash")
+    @logstash_plugin = @logstash.plugin_cli
   end
 
   if RbConfig::CONFIG["host_os"] == "linux"
@@ -55,6 +60,8 @@
           presence_check = @logstash_plugin.list(test_plugin)
           expect(presence_check.exit_code).to eq(1)
           expect(presence_check.stderr_and_stdout).to match(/ERROR: No plugins found/)
+
+          expect("logstash-filter-qatest").to_not be_installed_gem
         end
       end
 
@@ -92,6 +99,38 @@
       presence_check = @logstash_plugin.list(test_plugin)
       expect(presence_check.exit_code).to eq(1)
       expect(presence_check.stderr_and_stdout).to match(/ERROR: No plugins found/)
+
+      expect("logstash-filter-qatest").to_not be_installed_gem
+    end
+  end
+
+  context "plugins with unshared dependencies" do
+    let(:plugin_to_remove) {  }
+
+    it "successfully removes the plugin and its unshared dependencies" do
+      execute = @logstash_plugin.remove("logstash-integration-aws")
+
+      expect(execute.exit_code).to eq(0)
+      expect(execute.stderr_and_stdout).to match(/Successfully removed logstash-integration-aws/)
+
+      expect("logstash-integration-aws").to_not be_installed_gem
+
+      # known unshared dependencies, including transitive dependencies
+      aggregate_failures("known unshared dependencies") do
+        expect("aws-sdk-core").to_not be_installed_gem
+        expect("aws-sdk-s3").to_not be_installed_gem
+        expect("aws-sdk-kms").to_not be_installed_gem
+        expect("aws-sdk-cloudfront").to_not be_installed_gem
+        expect("aws-sdk-cloudwatch").to_not be_installed_gem
+        expect("aws-eventstream").to_not be_installed_gem
+        expect("aws-partitions").to_not be_installed_gem
+      end
+
+      # known shared dependencies
+      aggregate_failures("known shared dependencies") do
+        expect("concurrent-ruby").to be_installed_gem
+        expect("logstash-codec-json").to be_installed_gem
+      end
     end
   end
 
@@ -108,6 +147,8 @@
 
       expect(presence_check.exit_code).to eq(0)
       expect(presence_check.stderr_and_stdout).to match(/logstash-codec-json/)
+
+      expect("logstash-codec-json").to be_installed_gem
     end
   end
 
diff --git a/qa/integration/specs/cli/update_spec.rb b/qa/integration/specs/cli/update_spec.rb
new file mode 100644
index 00000000000..56ad75ec791
--- /dev/null
+++ b/qa/integration/specs/cli/update_spec.rb
@@ -0,0 +1,65 @@
+# Licensed to Elasticsearch B.V. under one or more contributor
+# license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright
+# ownership. Elasticsearch B.V. licenses this file to you under
+# the Apache License, Version 2.0 (the "License"); you may
+# not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+require_relative "../../framework/fixture"
+require_relative "../../framework/settings"
+require_relative "../../services/logstash_service"
+require_relative "../../framework/helpers"
+require_relative "pluginmanager_spec_helper"
+require "logstash/devutils/rspec/spec_helper"
+
+describe "CLI > logstash-plugin update" do
+
+  include_context "pluginmanager validation helpers"
+
+  before(:each) do
+    @fixture = Fixture.new(__FILE__)
+    @logstash = @fixture.get_service("logstash")
+    @logstash_plugin = @logstash.plugin_cli
+  end
+
+  context "upgrading a plugin" do
+    before(:each) do
+      aggregate_failures("precheck") do
+        expect("logstash-filter-qatest").to_not be_installed_gem
+      end
+      aggregate_failures("setup") do
+        execute = @logstash_plugin.install("logstash-filter-qatest", version: "0.1.0")
+
+        expect(execute.stderr_and_stdout).to match(/Installation successful/)
+        expect(execute.exit_code).to eq(0)
+
+        expect("logstash-filter-qatest-0.1.0").to be_installed_gem
+      end
+    end
+    it "upgrades the plugin and cleans the old one" do
+      execute = @logstash_plugin.update("logstash-filter-qatest")
+
+      aggregate_failures("command execution") do
+        expect(execute.stderr_and_stdout).to include("Updated logstash-filter-qatest 0.1.0 to 0.1.1")
+        expect(execute.exit_code).to eq(0)
+      end
+
+      installed = @logstash_plugin.list("logstash-filter-qatest", verbose: true)
+      expect(execute.exit_code).to eq(0)
+      expect(installed.stderr_and_stdout).to include("logstash-filter-qatest")
+
+      expect("logstash-filter-qatest-0.1.1").to be_installed_gem
+      expect("logstash-filter-qatest-0.1.0").to_not be_installed_gem
+    end
+  end
+end
\ No newline at end of file


Reviews:
user: login: donoghuc
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:08:27Z
user: login: donoghuc
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:09:17Z
user: login: donoghuc
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-03T17:10:16Z
user: login: yaauie
state: COMMENTED
author_association: MEMBER
submitted_at: 2025-03-04T14:22:22Z
user: login: donoghuc
body: Ah, I didnt even notice the duplicate versions 🤦‍♂️ that makes total sense. This truly is a great improvement! 
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-04T18:14:11Z

Review comments:
diff_hunk: @@ -152,4 +162,95 @@ def gem_in_lock_file?(pattern, lock_file)
       end
     end
   end
+
+  context "rubygems hosted plugin" do
+    include_context "pluginmanager validation helpers"
+    shared_examples("overwriting existing") do
+      # let(:plugin_name) { defined?(super) ? super() : fail }
+      # let(:existing_plugin_version) { defined?(super) ? super() : fail }
+      # let(:specified_plugin_version) { defined?(super) ? super() : fail }
+      before(:each) do
path: qa/integration/specs/cli/install_spec.rb
user: login: donoghuc
body: ```suggestion
      before(:each) do
```
Remove commented out bindings
created_at: 2025-03-03T17:08:27Z
author_association: MEMBER
diff_hunk: @@ -47,6 +47,31 @@ def remove_unused_locally_installed_gems!
     end
   end
 
+  def remove_orphan_dependencies!
+    locked_gem_names = ::Bundler::LockfileParser.new(File.read(LogStash::Environment::LOCKFILE)).specs.map(&:full_name).to_set
+    orphan_gem_specs = ::Gem::Specification.each
+                                           .reject(&:stubbed?) # skipped stubbed (uninstalled) gems
+                                           .reject(&:default_gem?) # don't touch jruby-included default gems
+                                           .reject{ |spec| locked_gem_names.include?(spec.full_name) }
+                                           .sort
+
+    inactive_plugins, orphaned_dependencies = orphan_gem_specs.partition { |spec| LogStash::PluginManager.logstash_plugin_gem_spec?(spec) }
+
+    # uninstall plugins first, to limit damage should one fail to uninstall
+    inactive_plugins.each { |plugin| uninstall_gem!("inactive plugin", plugin) }
+    orphaned_dependencies.each { |dep| uninstall_gem!("orphaned dependency", dep) }
+  end
+
+  def uninstall_gem!(desc, spec)
+    require "rubygems/uninstaller"
+    Gem::DefaultUserInteraction.use_ui(debug? ? Gem::DefaultUserInteraction.ui : Gem::SilentUI.new) do
+      Gem::Uninstaller.new(spec.name, version: spec.version, force: true, executables: true).uninstall
+    end
+    puts "cleaned #{desc} #{spec.name} (#{spec.version})"
+  rescue Gem::InstallError => e
+    fail "Failed to uninstall `#{spec.full_name}`"
path: lib/pluginmanager/command.rb
user: login: donoghuc
body: Shall we surface the error message or other details? 
created_at: 2025-03-03T17:09:17Z
author_association: MEMBER
diff_hunk: @@ -47,6 +47,31 @@ def remove_unused_locally_installed_gems!
     end
   end
 
+  def remove_orphan_dependencies!
+    locked_gem_names = ::Bundler::LockfileParser.new(File.read(LogStash::Environment::LOCKFILE)).specs.map(&:full_name).to_set
+    orphan_gem_specs = ::Gem::Specification.each
+                                           .reject(&:stubbed?) # skipped stubbed (uninstalled) gems
+                                           .reject(&:default_gem?) # don't touch jruby-included default gems
+                                           .reject{ |spec| locked_gem_names.include?(spec.full_name) }
+                                           .sort
+
+    inactive_plugins, orphaned_dependencies = orphan_gem_specs.partition { |spec| LogStash::PluginManager.logstash_plugin_gem_spec?(spec) }
+
+    # uninstall plugins first, to limit damage should one fail to uninstall
+    inactive_plugins.each { |plugin| uninstall_gem!("inactive plugin", plugin) }
+    orphaned_dependencies.each { |dep| uninstall_gem!("orphaned dependency", dep) }
+  end
+
+  def uninstall_gem!(desc, spec)
+    require "rubygems/uninstaller"
path: lib/pluginmanager/command.rb
user: login: donoghuc
body: Is this expensive to load on startup? Why do we wait until method is called to load implementation? 
created_at: 2025-03-03T17:10:16Z
author_association: MEMBER
diff_hunk: @@ -47,6 +47,31 @@ def remove_unused_locally_installed_gems!
     end
   end
 
+  def remove_orphan_dependencies!
+    locked_gem_names = ::Bundler::LockfileParser.new(File.read(LogStash::Environment::LOCKFILE)).specs.map(&:full_name).to_set
+    orphan_gem_specs = ::Gem::Specification.each
+                                           .reject(&:stubbed?) # skipped stubbed (uninstalled) gems
+                                           .reject(&:default_gem?) # don't touch jruby-included default gems
+                                           .reject{ |spec| locked_gem_names.include?(spec.full_name) }
+                                           .sort
+
+    inactive_plugins, orphaned_dependencies = orphan_gem_specs.partition { |spec| LogStash::PluginManager.logstash_plugin_gem_spec?(spec) }
+
+    # uninstall plugins first, to limit damage should one fail to uninstall
+    inactive_plugins.each { |plugin| uninstall_gem!("inactive plugin", plugin) }
+    orphaned_dependencies.each { |dep| uninstall_gem!("orphaned dependency", dep) }
+  end
+
+  def uninstall_gem!(desc, spec)
+    require "rubygems/uninstaller"
path: lib/pluginmanager/command.rb
user: login: yaauie
body: I think there are a couple reasons why I did it this way.

First, it follows the prior art in this source file.

But I think the bigger factor is that the plugin manager is invoked stand-alone, and is not typically a long-running process, so there's less of a stability benefit to eager-loading. I don't _believe_ that having the uninstaller present will have side-effects for the non-mutation `list` command, but I'd rather not take chances.
created_at: 2025-03-04T14:22:22Z
author_association: MEMBER

PR comments:
user: login: mergify[bot]
created_at: 2025-02-28T22:29:35Z
body: This pull request does not have a backport label. Could you fix it @yaauie? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:
* `backport-8./d` is the label to automatically backport to the `8./d` branch. `/d` is the digit.
* `backport-8.x` is the label to automatically backport to the `8.x` branch.
author_association: CONTRIBUTOR
user: login: mergify[bot]
created_at: 2025-02-28T22:29:36Z
body: `backport-8.x` has been added to help with the transition to the new branch `8.x`.
If you don't need it please use `backport-skip` label.
author_association: CONTRIBUTOR
user: login: donoghuc
created_at: 2025-03-03T17:54:04Z
body: Did you ever figure out why it is getting so confused about the `public_suffix` and `ruby-maven` gems?  I see messages for it when I build LS from this branch. Below is another example where i build logstash off `main` and removed `logstash-output-webhdfs` (that is a good plugin because previously removing it would leave webhdfs orphaned). Then i checked out your code and did a plugin install which triggered the cleanup of some gems that I think should actually be *not* removed (though it seems that they are not actually being removed which is curious). 

```
➜  logstash git:(e19f96a3a9) ✗ bin/logstash-plugin install logstash-output-webhdfs
Using system java: /Users/cas/.jenv/shims/java
Validating logstash-output-webhdfs
Resolving mixin dependencies
Installing logstash-output-webhdfs
Installation successful
cleaned orphaned dependency public_suffix (6.0.1)
cleaned orphaned dependency ruby-maven (3.9.3)
➜  logstash git:(e19f96a3a9) ✗ cat Gemfile.lock | grep public_suffix
      public_suffix (>= 2.0.2, < 7.0)
      public_suffix (> 4, < 6)
    public_suffix (5.1.1)
➜  logstash git:(e19f96a3a9) ✗ find . -name "*public_suffix*"
./tools/dependencies-report/build/resources/main/notices/public_suffix-NOTICE.txt
./tools/dependencies-report/src/main/resources/notices/public_suffix-NOTICE.txt
./vendor/bundle/jruby/3.1.0/specifications/public_suffix-5.1.1.gemspec
./vendor/bundle/jruby/3.1.0/cache/public_suffix-5.1.1.gem
./vendor/bundle/jruby/3.1.0/gems/public_suffix-5.1.1
./vendor/bundle/jruby/3.1.0/gems/public_suffix-5.1.1/lib/public_suffix.rb
./vendor/bundle/jruby/3.1.0/gems/public_suffix-5.1.1/lib/public_suffix
./vendor/bundle/jruby/3.1.0/gems/domain_name-0.6.20240107/data/public_suffix_list.dat
➜  logstash git:(e19f96a3a9) ✗ cat Gemfile.lock | grep ruby-maven
      ruby-maven-libs (~> 3, >= 3.8.9)
    ruby-maven-libs (3.9.9)
  ruby-maven-libs (~> 3, >= 3.9.6.1)
➜  logstash git:(e19f96a3a9) ✗ find . -name "*ruby-maven*"
./tools/dependencies-report/build/resources/main/notices/ruby-maven-libs-NOTICE.txt
./tools/dependencies-report/src/main/resources/notices/ruby-maven-libs-NOTICE.txt
./build/cache/index.rubygems.org%443/quick/Marshal.4.8/ruby-maven-3.9.3.gemspec
./vendor/bundle/jruby/3.1.0/specifications/ruby-maven-libs-3.9.9.gemspec
./vendor/bundle/jruby/3.1.0/cache/ruby-maven-libs-3.9.9.gem
./vendor/bundle/jruby/3.1.0/gems/ruby-maven-libs-3.9.9
```
author_association: MEMBER
user: login: yaauie
created_at: 2025-03-04T13:58:57Z
body: > Did you ever figure out why it is getting so confused about the `public_suffix` and `ruby-maven` gems?

I'll address these separately.

The `ruby-maven` gem isn't a part of the lock and _should_ be removed. The `ruby-maven-libs` is a dependency of logstash core, but from what I understand `ruby-maven` is a command-line wrapper _around_ `ruby-maven-libs` that we don't need and don't depend on. My best guess is that it is installed as a side-effect of being a _development_ dependency of `jar-dependencies` (which would happen if the plugin manager is invoked with `install --development`,  but I'm not finding an invocation of the gradle task `installDevelopmentGems` or the rake tasks `plugin:install-development-dependencies` or `plugin:install-base` that have that flag).

The `public_suffix` has _two_ versions installed, and the one that is not in the lock _should_ be removed. The cleanup removes the `6.0.1` that is _not_ in the lockfile, and leaves the locked `5.1.1` in-tact. I believe that version `6.0.1` was installed during the `bootstrap` rake task, but that the act of installing `logstash-input-twitter` which depends on `public_suffix (> 4, < 6)` deactivates it and installs the `5.1.1` that satisfies the dependency graph.

author_association: MEMBER
user: login: elastic-sonarqube[bot]
created_at: 2025-03-04T14:26:37Z
body: ## [![Quality Gate passed](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/checks/QualityGateBadge/passed-16px.png 'Quality Gate passed')](https://sonar.elastic.dev/dashboard?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203) **Quality Gate passed**  
Issues  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203&resolved=false) [0 New issues](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203&resolved=false)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/fixed-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&fixedInPullRequest=17203) [0 Fixed issues](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&fixedInPullRequest=17203)  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/accepted-16px.png '')](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203&issueStatuses=ACCEPTED) [0 Accepted issues](https://sonar.elastic.dev/project/issues?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203&issueStatuses=ACCEPTED)

Measures  
[![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/passed-16px.png '')](https://sonar.elastic.dev/security_hotspots?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203) [0 Security Hotspots](https://sonar.elastic.dev/security_hotspots?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203)  
![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/no-data-16px.png '') No data about Coverage  
![](https://raw.githubusercontent.com/SonarSource/sonarqube-static-resources/master/v104/common/no-data-16px.png '') No data about Duplication  
  
[See analysis details on SonarQube](https://sonar.elastic.dev/dashboard?id=elastic_logstash_AYm_nEbQaV3I-igkX1q9&pullRequest=17203)


author_association: NONE
user: login: elasticmachine
created_at: 2025-03-04T14:47:26Z
body: ## :green_heart: Build Succeeded
* [Buildkite Build](https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/2456)
* Commit: c85df441cc8d645c6d3a835ac9ca5402be213f26

### History
* :green_heart: [Build #2444](https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/2444) succeeded e74c4fc2abd35dafb89b709fad38fdb263bfe0dc

cc @donoghuc

<!--buildkite-pr-comment-logstash-pull-request-pipeline
{"builds":[{"buildStatus":{"state":"passed","success":true,"hasRetries":false,"hasNonPreemptionRetries":false},"url":"https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/2456","number":2456,"commit":"c85df441cc8d645c6d3a835ac9ca5402be213f26"},{"buildStatus":{"state":"passed","success":true,"hasRetries":false,"hasNonPreemptionRetries":false},"url":"https://buildkite.com/elastic/logstash-pull-request-pipeline/builds/2444","number":2444,"commit":"e74c4fc2abd35dafb89b709fad38fdb263bfe0dc"}],"number":2456}
buildkite-pr-comment-->
author_association: NONE
user: login: yaauie
created_at: 2025-03-04T19:50:42Z
body: @mergify backport 9.0
author_association: MEMBER
user: login: mergify[bot]
created_at: 2025-03-04T19:50:57Z
body: > backport 9.0

#### ✅ Backports have been created

<details>

* [#17221 Pluginmanager clean after mutate (backport #17203)](https://github.com/elastic/logstash/pull/17221) has been created for branch `9.0`

</details>


<!---
DO NOT EDIT
-*- Mergify Payload -*-
{"command": "backport 9.0", "conclusion": "success", "action_is_running": true}
-*- Mergify Payload End -*-
-->
author_association: CONTRIBUTOR