Repository: traefik/traefik (language: go)

Issue:
v2.9 not starting: Error in Go routine: runtime error: invalid memory address or nil pointer dereference
### Welcome!

- [X] Yes, I've searched similar issues on [GitHub](https://github.com/traefik/traefik/issues) and didn't find any.
- [X] Yes, I've searched similar issues on the [Traefik community forum](https://community.traefik.io) and didn't find any.

### What did you do?

I attempted to run the current `v2.9` tagged docker image.

### What did you see instead?

```
project-1  | time="2022-09-22T14:19:26Z" level=info msg="Configuration loaded from flags."
project-1  | time="2022-09-22T14:19:27Z" level=error msg="Error in Go routine: runtime error: invalid memory address or nil pointer dereference"
project-1  | time="2022-09-22T14:19:27Z" level=error msg="Error in Go routine: runtime error: invalid memory address or nil pointer dereference"
project-1  | time="2022-09-22T14:19:27Z" level=error msg="Stack: goroutine 85 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/traefik/traefik/v2/pkg/safe.defaultRecoverGoroutine({0x32d2be0?, 0x62c5880})\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:66 +0xa5\ngithub.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1.1()\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:56 +0x36\npanic({0x32d2be0, 0x62c5880})\n\truntime/panic.go:884 +0x212\ngithub.com/traefik/traefik/v2/pkg/provider/acme.(*Provider).resolveDomains.func1()\n\tgithub.com/traefik/traefik/v2/pkg/provider/acme/provider.go:400 +0x29b\ngithub.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1()\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:59 +0x5b\ncreated by github.com/traefik/traefik/v2/pkg/safe.GoWithRecover\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:53 +0x72\n"
project-1  | time="2022-09-22T14:19:27Z" level=error msg="Stack: goroutine 84 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x65\ngithub.com/traefik/traefik/v2/pkg/safe.defaultRecoverGoroutine({0x32d2be0?, 0x62c5880})\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:66 +0xa5\ngithub.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1.1()\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:56 +0x36\npanic({0x32d2be0, 0x62c5880})\n\truntime/panic.go:884 +0x212\ngithub.com/traefik/traefik/v2/pkg/provider/acme.(*Provider).resolveDomains.func1()\n\tgithub.com/traefik/traefik/v2/pkg/provider/acme/provider.go:400 +0x29b\ngithub.com/traefik/traefik/v2/pkg/safe.GoWithRecover.func1()\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:59 +0x5b\ncreated by github.com/traefik/traefik/v2/pkg/safe.GoWithRecover\n\tgithub.com/traefik/traefik/v2/pkg/safe/routine.go:53 +0x72\n"
```

### What version of Traefik are you using?

```
traefik             v2.9             6f3f4055952e   5 days ago     107MB
```
```
Version:      2.9.0-rc3
Codename:     banon
Go version:   go1.19.1
Built:        2022-09-16T14:06:50Z
OS/Arch:      linux/amd64
```

### What is your environment & configuration?

```yaml
image: traefik:v2.9
```


### If applicable, please paste the log output in DEBUG level

_No response_


Discussion / hints:
Hello,

can you provide your configuration (yaml, toml, docker-compose file, etc.)? and the debug log from the start of Traefik?
> Hello,

Hi
> can you provide your configuration (yaml, toml, docker-compose file, etc.)?

Unfortunately, not at this time due to the vast number of fields containing sensitive data that I would have to go over and redact.
I do use ACME with Let's Encrypt, which might be related (see below), so here is the part that deals with certificate resolvers:

```yaml
      - --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=SOMEPROVIDER
      - --certificatesresolvers.letsencrypt.acme.email=SOMEEMAIL@DOMAIN.TLD
      - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
```

> and the debug log from the start of Traefik?

Same for this one, but I did have a minute to take a quick peek into the debug log and the one thing that stood out to me:
During the startup, it iterates over all my services and domain names when logging ACME-related things. The runtime errors don't start until it starts logging about ACME. It basically goes from
> Trying to challenge certificate [...] ACME CA="https://acme-v02.api.letsencrypt.org/directory" [...]

over
> Looking for provided certificate(s) [...]

and then ends up with
> No ACME certificate generation required for domains [\"redacted.tld\"]." ACME CA="https://acme-v02.api.letsencrypt.org/directory" providerName=letsencrypt.acme routerName=someservice@docker rule="Host(`redacted.tld`)

This happens with all the configured domains and subdomains, and it's all interleaved with the aforementioned runtime errors.
Feels like it might indeed be ACME-related.

Fix patch:
diff --git a/pkg/provider/acme/provider.go b/pkg/provider/acme/provider.go
index 81a77ab5ba..be211ab043 100644
--- a/pkg/provider/acme/provider.go
+++ b/pkg/provider/acme/provider.go
@@ -397,7 +397,7 @@ func (p *Provider) resolveDomains(ctx context.Context, domains []string, tlsStor
 				return
 			}
 
-			err = p.addCertificateForDomain(dom, cert.Certificate, cert.PrivateKey, tlsStore)
+			err = p.addCertificateForDomain(dom, cert, tlsStore)
 			if err != nil {
 				logger.WithError(err).Error("Error adding certificate for domain")
 			}
@@ -431,7 +431,7 @@ func (p *Provider) watchNewDomains(ctx context.Context) {
 										return
 									}
 
-									err = p.addCertificateForDomain(dom, cert.Certificate, cert.PrivateKey, traefiktls.DefaultTLSStoreName)
+									err = p.addCertificateForDomain(dom, cert, traefiktls.DefaultTLSStoreName)
 									if err != nil {
 										logger.WithError(err).Error("Error adding certificate for domain")
 									}
@@ -468,7 +468,7 @@ func (p *Provider) watchNewDomains(ctx context.Context) {
 										return
 									}
 
-									err = p.addCertificateForDomain(dom, cert.Certificate, cert.PrivateKey, traefiktls.DefaultTLSStoreName)
+									err = p.addCertificateForDomain(dom, cert, traefiktls.DefaultTLSStoreName)
 									if err != nil {
 										logger.WithError(err).Error("Error adding certificate for domain")
 									}
@@ -535,7 +535,7 @@ func (p *Provider) watchNewDomains(ctx context.Context) {
 							domain.SANs = validDomains[1:]
 						}
 
-						err = p.addCertificateForDomain(domain, cert.Certificate, cert.PrivateKey, traefiktls.DefaultTLSStoreName)
+						err = p.addCertificateForDomain(domain, cert, traefiktls.DefaultTLSStoreName)
 						if err != nil {
 							logger.WithError(err).Error("Error adding certificate for domain")
 						}
@@ -660,11 +660,15 @@ func (p *Provider) removeResolvingDomains(resolvingDomains []string) {
 	}
 }
 
-func (p *Provider) addCertificateForDomain(domain types.Domain, certificate, key []byte, tlsStore string) error {
+func (p *Provider) addCertificateForDomain(domain types.Domain, crt *certificate.Resource, tlsStore string) error {
+	if crt == nil {
+		return nil
+	}
+
 	p.certificatesMu.Lock()
 	defer p.certificatesMu.Unlock()
 
-	cert := Certificate{Certificate: certificate, Key: key, Domain: domain}
+	cert := Certificate{Certificate: crt.Certificate, Key: crt.PrivateKey, Domain: domain}
 
 	certUpdated := false
 	for _, domainsCertificate := range p.certificates {
@@ -828,7 +832,7 @@ func (p *Provider) renewCertificates(ctx context.Context, renewPeriod time.Durat
 			continue
 		}
 
-		err = p.addCertificateForDomain(cert.Domain, renewedCert.Certificate, renewedCert.PrivateKey, cert.Store)
+		err = p.addCertificateForDomain(cert.Domain, renewedCert, cert.Store)
 		if err != nil {
 			logger.WithError(err).Error("Error adding certificate for domain")
 		}


Test patch: