Repository: traefik/traefik (language: go)

Issue:
Provide dnsChallenge credentials per resolver
### Do you want to request a *feature* or report a *bug*?

Feature

### What did you expect to see?

multiple resolvers is 🔥
but what if I have multiple accounts

something like this:
```
[certificatesResolvers.sample1.acme]
  email = "info@example.com"
  storage = "acme/acme.json"
  [certificatesResolvers.sample1.acme.dnsChallenge]
    provider = "cloudflare"
    <API_KEY> = <.......>
[certificatesResolvers.sample2.acme]
  email = "info2@example.com"
  storage = "acme/acme.json"
  [certificatesResolvers.sample2.acme.dnsChallenge]
    provider = "cloudflare"
    <API_KEY> = <.......>
```

use case: Traefik running multiple domains that belongs to different accounts on Cloudflare


Discussion / hints:
Hi @bstuff , thanks for reporting this use case. This issue is labelled as "proposal" as there should be a discussion on the topics below:

It looks like that implementing this would require the following challenge to be solved:

- Disparity of behavior between DNS providers: using token, API key, instance profiles or any other way for using account credentials depends on the provider. This could cause a lot of complexity.
- Some providers make it really hard for this. Example with AWS where the client has to manage credentials between API keys, instance profiles and IAM roles. Profiles should be used in this case, but only in this case.

Finally, please note that this proposal would lead to some changes in Lego (the library used for ACME usage in Traefik), not only on Traefik.
Hello, something is new for that proposal ?

I ask because i want setup traefik with multiple domains but i have multiple credentials :(
For now i am oblige to manage certificates with certbot externally to traefik.

Thanks.
May be a solution is to run the `lego` as a separated process? It would be really nice to have separated credentials per domain…

edit: or is there any way to reload bound certificates on-demand?
I have a specific use case for this, which I hope will bring interest to this feature request.

I share a server with someone, and we have two domains, one each. We use Traefik to redirect users to our respective containerized services and provide the respective certificates, which are both **wildcard** certs. 

It'd be a great value to add this feature to Traefik, since I think it matches its philosophy quite well.
Being able to configure secrets per provider would also be quite nice.

#5632 seems related
Here is an idealized `docker-compose.yml` that mimics the actual [DNS challenge user guide](https://docs.traefik.io/v2.0/user-guides/docker-compose/acme-dns/).

```
version: "3.3"

services:

  traefik:
    image: "traefik:v2.1"
    container_name: "traefik"
    command:
      #- "--log.level=DEBUG"
      #- "--api.insecure=true"
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      # Arguments related to generating "domain1.ex" certificates
      - "--certificatesresolvers.domain1_resolver.acme.dnschallenge=true"
      - "--certificatesresolvers.domain1_resolver.acme.email=postmaster@domain1.ex"
      - "--certificatesresolvers.domain1_resolver.acme.storage=/letsencrypt_domain1/acme.json"
      - "--certificatesresolvers.domain1_resolver.acme.dnschallenge.provider=_domain1_provider"
      # would be best to use secrets here
      - "--certificatesresolvers.domain1_resolver.acme.dnschallenge.provider.apikey=..."
      # Arguments related to generating "domain2.ex" certificates
      - "--certificatesresolvers.domain2_resolver.acme.dnschallenge=true"
      - "--certificatesresolvers.domain2_resolver.acme.email=postmaster@domain2.ex"
      - "--certificatesresolvers.domain2_resolver.acme.storage=/letsencrypt_domain2/acme.json"
      - "--certificatesresolvers.domain2_resolver.acme.dnschallenge.provider=_domain2_provider"
      # would be best to use secrets here
      - "--certificatesresolvers.domain2_resolver.acme.dnschallenge.provider.apikey=..."

    ports:
      - "80:80"
      - "443:443"

    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      # Bind mount storing certificates for "domain1.ex"
      - "/home/domain1_postmaster/lets_encrypt:/letsencrypt_domain1"
      # Bind mount storing certificates for "domain2.ex"
      - "/home/domain2_postmaster/lets_encrypt:/letsencrypt_domain2"

  whoami_domain1:
    image: "containous/whoami"
    container_name: "whoami_domain1"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami_domain1.rule=Host(`whoami.domain1.ex`)"
      - "traefik.http.routers.whoami_domain1.entrypoints=websecure"
      - "traefik.http.routers.whoami_domain1.tls.certresolver=domain1_resolver"

  whoami_domain2:
    image: "containous/whoami"
    container_name: "whoami_domain2"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami_domain2.rule=Host(`whoami.domain2.ex`)"
      - "traefik.http.routers.whoami_domain2.entrypoints=websecure"
      - "traefik.http.routers.whoami_domain2.tls.certresolver=domain2_resolver"
```
Edit: few inconsistencies
I'm wondering if it would be possible to ask traefik to update the env variables before a call into lego client. It is not ideal, because at the moment traefik does not even know that they exist, and it will have to. But the advantage of that approach is that lego won't need to be changed. Of course it will require synchronisation so no more than one challenge for a particular dns provider is running at the same time, but this seems doable. This could be an interim solution, until we can figure out how to do this better, it does not seem that there are a lot of options though.

Otherwise people have no choice but to run [two instances of traefik](https://community.containo.us/t/traefik-letsencrypt-dnschallenge-workaround-with-haproxy/6930).
Is there a solution in sight? Will there be a solution at all? Where can I find out more?
I have a similar need -- I am looking for options to leverage multiple godaddy accounts (several different API keys).

Here is a link to a thread I started in the community on it (almost identitical to @bstuff original post
https://community.traefik.io/t/traefik-let-s-encrypt-dns-challenge-with-multiple-instances-of-same-provider/7917

Any chance of getting this into the road map?  Right now it seems the only option is to run multiple copies of traefik (different docker servers).
any news here ?
Could a temp solution be to use "External Program" provider?
You could then write a wrapper around another dnsChallanger or your own challanger. Just extract the host and match it against a collection of credentials?
> Could a temp solution be to use "External Program" provider?
> You could then write a wrapper around another dnsChallanger or your own challanger. Just extract the host and match it against a collection of credentials?

Is it possible to use for example "certs bots" and provide the certificat to traefik ?
Any updates here?
No news for me ...
@ldez maybe you have some thoughts on how can this issue be resolved properly?
If so, can you please share them with community?
Maybe we would be able to bring you a good solution based on your thoughts.

Right now this issue seems to be abandoned and I'd like to bump it.
I hope this gets fixed soon. I'm stuck using let's encrypt for a whole set of subdomains and I can't assign a new acme provider without dns challenge. I don't want to try to run two traefik instances and combine them or something. 

```yaml
- --certificatesresolvers.resolverA.acme.caServer=
- --certificatesresolvers.resolverA.acme.email=
- --certificatesresolvers.resolverA.acme.storage=/acme.json
- --certificatesresolvers.resolverA.acme.eab.kid=
- --certificatesresolvers.resolverA.acme.eab.hmacEncoded=

# cloudflare dns challenge
- --certificatesresolvers.resolverA.acme.dnsChallenge.provider=cloudflare
# this!!! it should be local not global!
- --certificatesresolvers.resolverA.acme.dnsChallenge.env.CF_API_EMAIL=
- --certificatesresolvers.resolverA.acme.dnsChallenge.env.CF_API_KEY=
```

I don't see this being too hard to implement? When registering `resolverA`, instead of getting `CF_API_EMAIL` and `CF_API_KEY` from the process environment, just pick it up from the config it's currently in: `certificatesresolvers.resolverA.acme.dnsChallenge.env`

I would add it in myself but I've never used go before.
Okay I see the problem. This is where it fetches things:
https://github.com/go-acme/lego/blob/8e7bba485fe98dcb6f116eb9f524cd37312540ac/providers/dns/cloudflare/cloudflare.go#L70
It's in a different library hmm.
Lego calls a function named `GetOrFile` here:
https://github.com/go-acme/lego/blob/8e7bba485fe98dcb6f116eb9f524cd37312540ac/platform/config/env/env.go#L146

Which will look for CF_API_EMAIL_FILE and read it's contents. This sounds so silly, but if you were to create a temporary file and write to it before initializing the dns provider... haha nevermind...

It doesn't look there's an easy fix other than working together with the lego people to add a new function to replace here:
https://github.com/traefik/traefik/blob/ce2e02b6904329d4c62a2f04e2707869ea525271/pkg/provider/acme/provider.go#L269

`NewDNSChallengeProviderByNameWithCustomEnv`..?

I found this issue after a discussion with a user who is impacted by this.

Not sure why recent comments that share community-based investigations toward a solution (for example those by @makitsune) are being marked as off-topic, especially while repeated comments that only ask for updates remain unflagged. This community-driven sharing is what makes open source productive and useful. I think they should be un-flagged to reveal useful information and add context to what's happening in the code.

Indeed, as @makitsune shows, the underlying ACME library, lego, reads DNS provider credentials from environment variables. This makes it difficult/impossible to configure different provider credentials in a single environment, at least with the current implementation of lego.

While my comment will probably get hidden, I wanted to propose a solution to be considered anyway, with some background.

About a year and a half ago, I [raised an issue in lego similar to this one](https://github.com/go-acme/lego/issues/1054), and later [proposed a change that treated env vars as optional defaults instead of as the only way to configure the credentials](https://github.com/go-acme/lego/pull/1058).

I admit I wasn't able to design a satisfactory solution without making other major changes to the code base, so I let the PR be closed. My solution instead was to write [a leaner ACME library named `acmez`](https://github.com/mholt/acmez) that has a completely different API. It [can be configured with a DNS challenge solver](https://pkg.go.dev/github.com/mholt/acmez#Client) that gets its credentials without the need for environment variables.

The [DNS solver I implemented is in CertMagic](https://pkg.go.dev/github.com/caddyserver/certmagic#DNS01Solver), and it's an exported type so it can be used by any library, including lego. Notice that the DNSProvider field is a type that uses [any `libdns` implementation](https://pkg.go.dev/github.com/caddyserver/certmagic#ACMEDNSProvider):

```go
type ACMEDNSProvider interface {
	libdns.RecordAppender
	libdns.RecordDeleter
}
```

[`libdns`](https://github.com/libdns/libdns) is a set of interfaces implemented by a [collection of libraries](https://github.com/libdns) that manipulates records on various DNS providers. In this case, ACME challenges require appending and deleting records, hence the above interface. There are already a fair number of implementations, with more on the way -- and they're relatively easy to implement new ones.

(We know this works because this is what Caddy 2 uses to support multiple DNS credentials per config.)

tl;dr - Using `certmagic.DNS01Solver` along with `libdns` libraries (and possibly `acmez` which is already compatible with them) can solve the issue at hand since these APIs do not require the use of the environment. It may require some changes in lego, but the changes in Traefik -- if any at all -- should be minimal.
@mholt excellent suggestion and approach.  Hopefully the traefik team is interested in it.
To illustrate more concretely, here is what a solution could look like (again, this would have to go inside lego, probably -- and I've omitted details like getting the ACME account and private key):

```go
client := acmez.Client{
	Client: &acme.Client{
		Directory: "https://...",
	},
	ChallengeSolvers: map[string]acmez.Solver{
		acme.ChallengeTypeDNS01: &certmagic.DNS01Solver{
			// notice that credentials are not provided by env vars (but could be by using os.Getenv)
			DNSProvider: &cloudflare.Provider{
				APIToken: "secret",
			},
		},	
	},
}

ctx := context.Background()

// put your domains here that are associated with the DNS credentials above
domains := []string{"example.com"}

certs, err := client.ObtainCertificate(ctx, acmeAccount, certPrivateKey, domains)
```

Notice how DNS providers can be plugged into the solver config easily, and doesn't require the use of env vars, but does allow them if you want. An approach/API like this could resolve the issue because you can specify different DNS credentials for different domains in your Traefik config. But I do think actual dev work would need to be tracked in [go-acme/lego](https://github.com/go-acme/lego).
The suggestions made by @makitsune and @mholt are based on the same assumption that all DNS providers can be configured via environment variables or with a simple configuration structure but this assumption missed that there are several ways to configure DNS providers and some of them don't use environment variables/configuration structure but elements based on the system configuration.

The problem has been already explained here:
- https://github.com/go-acme/lego/issues/1054#issuecomment-583901705
- https://github.com/traefik/traefik/issues/5472#issuecomment-534908109

Be sure that I know really well the problem, I have already spent a lot of time trying to find a solution, and I still looking for a viable approach.

> I have already spent time finding a way to have several configurations for a single provider, but it is quite difficult because of non-homogeneous providers' behavior.
https://github.com/go-acme/lego/issues/1054#issuecomment-583919285

If you want to discuss more feel free to open an issue on [lego](https://github.com/go-acme/lego).

Question, have you seen the approach that [Posh-Acme](https://github.com/rmbolger/Posh-ACME)?  The model he took seems to work well with almost any DNS based system.
Maybe this sounds silly and might not be possible in go, but when using node sounds reasonable: to spin up a new thread/instance with the resolver defined environment variables and close it when its done.
I know the other ACME client implementations and in Go we have Go routines (~threads) but it's neither the problem nor the solution.

I will repeat: some DNS providers don't use environment variables/configuration structure but elements **based on the system**.

Please, if you want to discuss more feel free to open an issue on [lego](https://github.com/go-acme/lego).
Can you help us understand what you mean by that? Like what are examples of DNS libraries that can only be configured by external settings and not by code?
Please, if you want to discuss more feel free to open an issue on [lego](https://github.com/go-acme/lego).
If going by the replies there seems no option other than to consolidate all domains into one authoritative DNS provider (Cloudflare in my case) and then use `DNS-01`. This solution is not possible as some clients have their own CF account and are not comfortable delegating their domain to someone else's CF account. I would be grateful if the community could provide a feasible workaround.

Fix patch:
diff --git a/docs/content/https/acme.md b/docs/content/https/acme.md
index f423b23dbb..b6e50bcf80 100644
--- a/docs/content/https/acme.md
+++ b/docs/content/https/acme.md
@@ -294,6 +294,12 @@ Use the `DNS-01` challenge to generate and renew ACME certificates by provisioni
     LEGO_DISABLE_CNAME_SUPPORT=true
     ```
 
+!!! warning "Multiple DNS Challenge provider"
+    
+    Multiple DNS challenge provider are not supported with Traefik, but you can use `CNAME` to handle that.
+    For example, if you have `example.org` (account foo) and `example.com` (account bar) you can create a CNAME on `example.org` called `_acme-challenge.example.org` pointing to `challenge.example.com`.
+    This way, you can obtain certificates for `example.com` with the `foo` account.
+
 !!! important
     A `provider` is mandatory.
 


Test patch: