Repository: hashicorp/nomad (Go)

Issue:
Nomad executes any file in `/plugins/` disregarding the plugin contract.
### Nomad version
tip

### Operating system and Environment details
Unix

### Issue

The advertised[ Nomad contract is](https://developer.hashicorp.com/nomad/docs/configuration/plugin) seems to be:

```
The name of the plugin is the plugin's executable name relative to to the plugin_dir. If the plugin has a suffix, such as .exe, this should be omitted.
```

Consequently, if the `plugins` directory looks as follows:

```
\- plugins
  \- foo.exe
  \- bar.exe
```

And the config looks as follows:

```
[...]
  "plugin": {
    "foo": {...}.
    [...]
   }
[...]
```

I would expect `foo.exe` to be used as a plugin and `bar.exe` ignored since it is not mentioned within the configuration at all. However, it seems that both `foo.exe` and `bar.exe` will be executed and used for plugin configuration. Even worse, if both` bar.exe` and `foo.exe` correspond to a driver whose fingerprinting endpoints return the same name and version, it is non-deterministic which one of the binaries will actually be used as the driver.

### Root cause.

When Nomad starts up, it will load plugins and fingerprint them. The rough codepath is:

1. Setup plugins: https://github.com/hashicorp/nomad/blob/main/command/agent/plugins.go#L15
2. New plugin loader: https://github.com/hashicorp/nomad/blob/main/helper/pluginutils/loader/loader.go#L115
3. Init plugin loader: https://github.com/hashicorp/nomad/blob/main/helper/pluginutils/loader/init.go#L57

Now, this has the following logic which goes against the contract described above:

4. `ls`  the plugin directory: https://github.com/hashicorp/nomad/blob/main/helper/pluginutils/loader/init.go#L68
5. execute  all files in it: https://github.com/hashicorp/nomad/blob/main/helper/pluginutils/loader/init.go#L322
6. Use the response of the plugin to get the name of it: https://github.com/hashicorp/nomad/blob/main/helper/pluginutils/loader/init.go#L279-L302

So in other words: the name of the binary is only used to derive the config json for the driver, not to control whether or not the driver should be executed in the first place.

### Reproduction steps

The simplest repro is to have an unrelated, non-executable file within `plugins` and observe that Nomad will eagerly execute it (or rather attempt and fail) despite the `config` not mentioning the file as a plugin.

The non-deterministic choice of the version of the config is also simple to reproduce. Take any functioning task driver and cause it to fail on first task start. Copy the task driver into the `plugins/` directory 10 times and add a single working copy. Observe that the correct driver is used around 9% of the time.


Discussion:
Unfortunately at this point I'm afraid the behavior needs to be treated as correct and the docs updated to reflect that.

The intent is that all plugins in the plugin directory are loaded by default regardless of explicit configuration. To change that to require explicit configuration now may break long existing Nomad clusters which rely on the autodetection mechanism.

Unprivileged write access to a plugin directory must be prevented. If you think this is insufficiently safe let me know. We could always warn users in upcoming releases and schedule the breaking change in the future. That's just a lot of effort when unprivileged write access to the plugin directory would still likely allow for attacks.

Non-determinism around which plugin is selected is a bug, but I'm unsure what the proper fix is. Lexicographically selecting the first one or crashing with an error both seem defensible options: the former assumes it doesn't matter and the latter assumes it was an operator mistake that needs correcting.

> The intent is that all plugins in the plugin directory are loaded by default regardless of explicit configuration. To change that to require explicit configuration now may break long existing Nomad clusters which rely on the autodetection mechanism.

What if we made this behavior configurable with the default reflecting the old behavior?

> Unprivileged write access to a plugin directory must be prevented. If you think this is insufficiently safe let me know. 

Agreed -- Having access to this directory already allows one to mimick a registered plugin. I think I am more concerned with the ease with which one may add non-determinism mentioned, but your point about maintaining an old observed behavior resonates with me
I'm going to lock this issue because it has been closed for _120 days_ ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.



Patch:
diff --git a/.changelog/18530.txt b/.changelog/18530.txt
new file mode 100644
index 00000000000..2a26b16b3a0
--- /dev/null
+++ b/.changelog/18530.txt
@@ -0,0 +1,3 @@
+```release-note:breaking-change
+agent: Plugins stored within the `plugin_dir` will now only be executed when they have a corresponding `plugin` configuration block. Any plugin found without a corresponding configuration block will be skipped.
+```
diff --git a/helper/pluginutils/loader/init.go b/helper/pluginutils/loader/init.go
index 7d3003627db..9bcbc6081aa 100644
--- a/helper/pluginutils/loader/init.go
+++ b/helper/pluginutils/loader/init.go
@@ -264,10 +264,15 @@ func (l *PluginLoader) fingerprintPlugins(plugins []os.FileInfo, configs map[str
 	fingerprinted := make(map[PluginID]*pluginInfo, len(plugins))
 	for _, p := range plugins {
 		name := cleanPluginExecutable(p.Name())
+
+		// Use the cleaned plugin name to check whether it is configured by the
+		// operator for use. If it is not, skip loading it and log a message, so
+		// operators can easily see this.
 		c, ok := configs[name]
 		if !ok {
-			// COMPAT(1.7): Skip executing unconfigured plugins in 1.8 or later.
-			l.logger.Warn("plugin not referenced in the agent configuration file, future versions of Nomad will not load this plugin until the agent configuration is updated", "plugin", name)
+			l.logger.Warn("plugin not referenced in the agent configuration file, loading skipped",
+				"plugin", name)
+			continue
 		}
 		info, err := l.fingerprintPlugin(p, c)
 		if err != nil {
diff --git a/website/content/docs/upgrade/upgrade-specific.mdx b/website/content/docs/upgrade/upgrade-specific.mdx
index b21a72abea8..7eb21db5e9f 100644
--- a/website/content/docs/upgrade/upgrade-specific.mdx
+++ b/website/content/docs/upgrade/upgrade-specific.mdx
@@ -36,6 +36,13 @@ Drivers with the `RemoteTasks` capability will no longer be detached in the even
 the allocation is lost, nor will remote tasks be detached when a node is drained.
 Workloads running as remote tasks should be migrated prior to upgrading.
 
+#### Loading Binaries from `plugin_dir` Without Configuration
+
+Plugins stored within the [`plugin_dir`](/nomad/docs/configuration#plugin_dir)
+will now only be loaded when they have a corresponding
+[`plugin`](/nomad/docs/configuration/plugin) block in the agent configuration
+file.
+
 ## Nomad 1.9.5
 
 #### CNI plugins


Test patch:
diff --git a/helper/pluginutils/loader/loader_test.go b/helper/pluginutils/loader/loader_test.go
index 939ee7b7104..5b0f4ce01ab 100644
--- a/helper/pluginutils/loader/loader_test.go
+++ b/helper/pluginutils/loader/loader_test.go
@@ -102,8 +102,9 @@ func TestPluginLoader_External(t *testing.T) {
 	ci.Parallel(t)
 	require := require.New(t)
 
-	// Create two plugins
-	plugins := []string{"mock-device", "mock-device-2"}
+	// Create three plugins. Only two will be referenced within the agent config
+	// meaning the third should not be loaded.
+	plugins := []string{"mock-device", "mock-device-2", "mock-device-3"}
 	pluginVersions := []string{"v0.0.1", "v0.0.2"}
 	h := newHarness(t, plugins)
 
@@ -133,6 +134,8 @@ func TestPluginLoader_External(t *testing.T) {
 	require.NoError(err)
 
 	// Get the catalog and assert we have the two plugins
+	//
+	// Note: mock-device-3 is ignored because it does not have a related config.
 	c := l.Catalog()
 	require.Len(c, 1)
 	require.Contains(c, base.PluginTypeDevice)