Repository: traefik/traefik (language: go)

Issue:
Plugins disabled in Kubernetes
### Welcome!

- [X] Yes, I've searched similar issues on [GitHub](https://github.com/traefik/traefik/issues) and didn't find any.
- [X] Yes, I've searched similar issues on the [Traefik community forum](https://community.traefik.io) and didn't find any.

### What did you do?

I'm trying to enable and install plugins.

### What did you see instead?

```
time="2023-05-25T14:36:04Z" level=error msg="Plugins are disabled because an error has occurred." error="mkdir plugins-storage: read-only file system"
```

In the configuration snippet below, I have mounted an `emtpyDir` volume in `/usr/local/bin`, because the documentation states that the plugins are expected to be found within the same directory as the binary. Note that this error occurs also if there is no volume mount at all, or if it is pointed at any arbitrary location.

### What version of Traefik are you using?

```sh
~ $ traefik version
Version:      2.10.1
Codename:     saintmarcelin
Go version:   go1.20.3
Built:        2023-04-27T14:52:35Z
OS/Arch:      linux/amd64
```

```
image: docker.io/traefik:v2.10
```

### What is your environment & configuration?

```sh
--accesslog 
--api.dashboard 
--api.insecure 
--entrypoints.metrics.address=:9100/tcp 
--entrypoints.traefik.address=:9000/tcp 
--entryPoints.web.address=:8000/tcp 
--entrypoints.web.forwardedHeaders.insecure 
--entrypoints.web.ProxyProtocol.insecure 
--entrypoints.websecure.address=:8443/tcp 
--entrypoints.websecure.http.tls=true 
--global.checknewversion=false 
--global.sendanonymoususage 
--log.level=INFO 
--metrics 
--metrics.prometheus 
--metrics.prometheus.entrypoint=metrics 
--ping 
--providers.kubernetescrd 
--providers.kubernetesingress 
--providers.kubernetesgateway 
--experimental.kubernetesgateway 
--experimental.plugins.geoBlock.modulename=github.com/PascalMinder/geoblock 
--experimental.plugins.geoBlock.version=v0.2.5 
--experimental.plugins.addTraceId.modulename=github.com/trinnylondon/traefik-add-trace-id 
--experimental.plugins.addTraceId.version=v0.1.5 
--experimental.plugins.correlationId.modulename=github.com/soulbalz/correlationid 
--experimental.plugins.correlationId.version=v1.0.0 
--tracing.zipkin 
--tracing.zipkin.httpEndpoint=http://jaeger-collector.observability:9411/api/v2/spans 
--certificatesresolvers.http.acme.email=REDACTED
--certificatesresolvers.http.acme.storage=/data/acme.json 
--certificatesresolvers.http.acme.httpchallenge.entrypoint=web 
--certificatesresolvers.dreamhost.acme.dnschallenge.provider=dreamhost 
--certificatesresolvers.dreamhost.acme.email=REDACTED
--certificatesresolvers.dreamhost.acme.storage=/data/acme.json 
--certificatesresolvers.linode.acme.dnschallenge.provider=linode 
--certificatesresolvers.linode.acme.email=REDACTED
--certificatesresolvers.linode.acme.storage=/data/acme.json
```

Add more configuration information here.

Volumes:
```yaml
          volumeMounts:
            - name: data
              mountPath: /data
            - name: plugins
              mountPath: /usr/local/bin/plugins-storage
            - name: tmp
              mountPath: /tmp
      volumes:
        - name: data
          emptyDir: {}
        - name: plugins
          emptyDir: {}
        - name: tmp
          emptyDir: {}
```

### If applicable, please paste the log output in DEBUG level

_No response_


Discussion / hints:
Hello,

> mkdir plugins-storage: read-only file system

Your volume is read-only then the plugins cannot be stored.

For help on your configuration, please join our [Community Forum](https://community.traefik.io/) and reach out to us on the [Traefik v2 section](https://community.traefik.io/c/traefik/traefik-v2). 


Sorry, but this issue is not resolved.

1. The software's error message doesn't provide anything remotely resembling useful information on troubleshooting the plugin directory that could not be created. What would be useful is knowing _in which directory_ "plugins-storage" is trying to be created.

2. The software erroneously attempts to create a directory that has already been provided.

3. The software does not provide a way to configure a directory to store the plugins, so that, in this specific case, when a read-only container volume is used, a writable directory can be provided.

These can all be considered bugs.
Hello @paulyhedral,

We re-open the issue to dig a bit deeper and have more thoughts about the solutions we could bring (better error message, better documentation, etc).

Moreover, we would love community support to address it.  

If you or another community member would like to build it, let us know and we will work with you to make sure you have all the information required so that it can be merged. 
> The software's error message doesn't provide anything remotely resembling useful information on troubleshooting the plugin directory that could not be created. 

We can try to see if we can improve this message.

> The software erroneously attempts to create a directory that has already been provided.

Traefik tries to create the directory and it doesn't create it if it already exists with the right permissions.

> The software does not provide a way to configure a directory to store the plugins, so that, in this specific case, when a read-only container volume is used, a writable directory can be provided.

The path to the directory is not configurable by design.


Fix patch:
diff --git a/cmd/traefik/plugins.go b/cmd/traefik/plugins.go
index f94535f925..c2b8478193 100644
--- a/cmd/traefik/plugins.go
+++ b/cmd/traefik/plugins.go
@@ -35,12 +35,12 @@ func initPlugins(staticCfg *static.Configuration) (*plugins.Client, map[string]p
 		var err error
 		client, err = plugins.NewClient(opts)
 		if err != nil {
-			return nil, nil, nil, err
+			return nil, nil, nil, fmt.Errorf("unable to create plugins client: %w", err)
 		}
 
 		err = plugins.SetupRemotePlugins(client, staticCfg.Experimental.Plugins)
 		if err != nil {
-			return nil, nil, nil, err
+			return nil, nil, nil, fmt.Errorf("unable to set up plugins environment: %w", err)
 		}
 
 		plgs = staticCfg.Experimental.Plugins
diff --git a/pkg/plugins/client.go b/pkg/plugins/client.go
index e891da5c49..4fcdd1ab40 100644
--- a/pkg/plugins/client.go
+++ b/pkg/plugins/client.go
@@ -264,7 +264,7 @@ func (c *Client) unzipArchive(pName, pVersion string) error {
 	for _, f := range archive.File {
 		err = unzipFile(f, dest)
 		if err != nil {
-			return err
+			return fmt.Errorf("unable to unzip %s: %w", f.Name, err)
 		}
 	}
 
@@ -283,12 +283,17 @@ func unzipFile(f *zipa.File, dest string) error {
 	p := filepath.Join(dest, pathParts[1])
 
 	if f.FileInfo().IsDir() {
-		return os.MkdirAll(p, f.Mode())
+		err = os.MkdirAll(p, f.Mode())
+		if err != nil {
+			return fmt.Errorf("unable to create archive directory %s: %w", p, err)
+		}
+
+		return nil
 	}
 
 	err = os.MkdirAll(filepath.Dir(p), 0o750)
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to create archive directory %s for file %s: %w", filepath.Dir(p), p, err)
 	}
 
 	elt, err := os.OpenFile(p, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, f.Mode())
@@ -347,7 +352,7 @@ func (c *Client) WriteState(plugins map[string]Descriptor) error {
 
 	mp, err := json.MarshalIndent(m, "", "  ")
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to marshal plugin state: %w", err)
 	}
 
 	return os.WriteFile(c.stateFile, mp, 0o600)
@@ -361,10 +366,15 @@ func (c *Client) ResetAll() error {
 
 	err := resetDirectory(filepath.Join(c.goPath, ".."))
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to reset plugins GoPath directory %s: %w", c.goPath, err)
+	}
+
+	err = resetDirectory(c.archives)
+	if err != nil {
+		return fmt.Errorf("unable to reset plugins archives directory: %w", err)
 	}
 
-	return resetDirectory(c.archives)
+	return nil
 }
 
 func (c *Client) buildArchivePath(pName, pVersion string) string {
@@ -374,12 +384,12 @@ func (c *Client) buildArchivePath(pName, pVersion string) string {
 func resetDirectory(dir string) error {
 	dirPath, err := filepath.Abs(dir)
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to get absolute path of %s: %w", dir, err)
 	}
 
 	currentPath, err := os.Getwd()
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to get the current directory: %w", err)
 	}
 
 	if strings.HasPrefix(currentPath, dirPath) {
@@ -388,10 +398,15 @@ func resetDirectory(dir string) error {
 
 	err = os.RemoveAll(dir)
 	if err != nil {
-		return err
+		return fmt.Errorf("unable to remove directory %s: %w", dirPath, err)
 	}
 
-	return os.MkdirAll(dir, 0o755)
+	err = os.MkdirAll(dir, 0o755)
+	if err != nil {
+		return fmt.Errorf("unable to create directory %s: %w", dirPath, err)
+	}
+
+	return nil
 }
 
 func computeHash(filepath string) (string, error) {
diff --git a/pkg/plugins/plugins.go b/pkg/plugins/plugins.go
index fdfb9fbcea..6888ceea4c 100644
--- a/pkg/plugins/plugins.go
+++ b/pkg/plugins/plugins.go
@@ -21,7 +21,7 @@ func SetupRemotePlugins(client *Client, plugins map[string]Descriptor) error {
 
 	err = client.CleanArchives(plugins)
 	if err != nil {
-		return fmt.Errorf("failed to clean archives: %w", err)
+		return fmt.Errorf("unable to clean archives: %w", err)
 	}
 
 	ctx := context.Background()
@@ -32,27 +32,27 @@ func SetupRemotePlugins(client *Client, plugins map[string]Descriptor) error {
 		hash, err := client.Download(ctx, desc.ModuleName, desc.Version)
 		if err != nil {
 			_ = client.ResetAll()
-			return fmt.Errorf("failed to download plugin %s: %w", desc.ModuleName, err)
+			return fmt.Errorf("unable to download plugin %s: %w", desc.ModuleName, err)
 		}
 
 		err = client.Check(ctx, desc.ModuleName, desc.Version, hash)
 		if err != nil {
 			_ = client.ResetAll()
-			return fmt.Errorf("failed to check archive integrity of the plugin %s: %w", desc.ModuleName, err)
+			return fmt.Errorf("unable to check archive integrity of the plugin %s: %w", desc.ModuleName, err)
 		}
 	}
 
 	err = client.WriteState(plugins)
 	if err != nil {
 		_ = client.ResetAll()
-		return fmt.Errorf("failed to write plugins state: %w", err)
+		return fmt.Errorf("unable to write plugins state: %w", err)
 	}
 
 	for _, desc := range plugins {
 		err = client.Unzip(desc.ModuleName, desc.Version)
 		if err != nil {
 			_ = client.ResetAll()
-			return fmt.Errorf("failed to unzip archive: %w", err)
+			return fmt.Errorf("unable to unzip archive: %w", err)
 		}
 	}
 


Test patch: