Repository: traefik/traefik (language: go)

Issue:
Support Failover Load-balancing
### What version of Traefik are you using (`traefik version`)?
v1.1.2

### What is your environment & configuration (arguments, toml...)?
linux, simple file backend

```toml
[backends]
  [backends.testing]
    [backends.testing.servers.server1]
    url = "http://primary:80"
    weight = 1
    [backends.testing.servers.server2]
    url = "http://backup:80"
    weight = 1
```

### What did you do?

I want to do load balancing simple active passive. however i want to use server only as backup ( in case of failure of network issues)


### What did you expect to see?
Wanted to see a simple case working where failover is supported instead of load-balancing

### What did you see instead?
Couldn't figure out how it will work by reading documentation N times. 

The use case is simple i have 2 server in single backend, one primary and one backup. However i would like to sent traffic to backup  only when primary is not working ( tcp connection could not be made).

I have tested some odd forms, where i add some abnormally high weight to primary and set retry to yes. In this case if primary is not responding,  retry again chooses the  primary  instead of backup.  So,  retry should be skipping the primary server toactually retry. This is broken. 


Can someone help me  to configure such as a case ?


Discussion / hints:
I am having the exact same problem. I want to deploy Traefik to fail-over to a different zone (e.g a different data-center).

It actually seems to compound the problem if the weights are set high as it appears that the weights consume retry.attempts.

If I add 8 servers (5 primary, 3 secondary) but all with equal weights, then take the primary ones offline it continues to field requests. But requests will also go to the backup servers.

If I set a higher weight (10) on the primary servers, then take them offline it seems like the amount of retries is "spent" by the weights of the primaries that its trying, so it never gets chance to try the backup servers. (well actually 3/8 requests succeed, as it lands on them first).

If I set the primary weight as 2 and the secondary as 1, then take the primaries offline, the ratio goes to 3/4 successful. The attempts automatically is set to 7, so if it lands on the first server, all 7 retries are spent on the weights of the 5 failed servers. All of the others work.

Lastly if I set the primary weight considerably higher (1000) and a high retry count 10000, and primaries offline, it 50 seconds to complete the first request.

If I was serving a static file then I could set the primary weight to 50 and secondaries to 1 then set 1000 as the retry count, and maybe drr strategy would help. The only problem is if there is an application level failure I have just DoS'd my own system, so no real solution here.

Would love to hear what I am doing wrong, or how I can configure this better. Thanks.
A HealthCheck feature was added in v1.2 by way of https://github.com/containous/traefik/pull/918 and https://github.com/containous/traefik/pull/1132

https://github.com/containous/traefik/blob/master/docs/basics.md#backends
Is there now a possible configuration for an active/passive failover configuration in v2 without setting the weight on a very high value?
It would be really nice to have this! Traefik rocks, but I will probably have to choose another solution given this is not supported.
Can someone address this issue please?
I am in need of a similar feature.

An example I gave in #6856 is of an AWS Target group, in that if there are no healthy services it'll route to all registered instances.

Currently if all instances fail a Consul health check, the route is completely removed. 
I've read all the actual mechanisms available and wondering about how this could be implemented:

* [CircuitBreaker](https://docs.traefik.io/middlewares/circuitbreaker/) with a customizable `Fallback mechanism`
* Supporting a Fallback mechanism within [LoadBalancing](https://docs.traefik.io/routing/services/#load-balancing)
* [WeightedRoundRobin](https://docs.traefik.io/routing/services/#weighted-round-robin-service) with special weight (e.g. a negative integer weight like `-1` which will only be used when the positive weights like `1` and `3` are not available.

> I've read all the actual mechanisms available and wondering about how this could be implemented:
> 
> * [CircuitBreaker](https://docs.traefik.io/middlewares/circuitbreaker/) with a customizable `Fallback mechanism`
> * Supporting a Fallback mechanism within [LoadBalancing](https://docs.traefik.io/routing/services/#load-balancing)
> * [WeightedRoundRobin](https://docs.traefik.io/routing/services/#weighted-round-robin-service) with special weight (e.g. a negative integer weight like `-1` which will only be used when the positive weights like `1` and `3` are not available.



I think the Fallback mechanism within LoadBalancing itself or the WeightedRoundRobin with special weight are the best ideas here. I want to be able to load a static page if my downstream servers are having issues. I can't seem to find a way to do this with Traefik in its current form.

Would a weight of 0 or -1 not be enough? Simply meaning: never use this server, unless the other(s) fail ?
The weight in `RoundRobin` is already defined, I would not try to mix that.

As a user, I would prefer to have the weight within `LoadBalancer`. Maybe it would be even possible to work with positive numbers here - so you could do things like

`5`  - (main service)
`5`  - (second main service)
`3`  - (fallback service - optional)
`1`  - (maintenance site)

Can we push this a little bit more? The first request was almost 4 years ago... There were already good solutions here. Can this be assigned to someone in the Traefik team?
Any update on this request please? 
Traefik is leap and bounds better than any reverse proxy solution available. Yet it is missing something as simple and straightforward as Active/Passive Configuration. 
Something which Nginx achieves using simple backup keyword against a server 

upstream backend {
    server backend1.example.com;
    server backup1.example.com  backup;
}

The definition of backup as per documentation here [https://nginx.org/en/docs/http/ngx_http_upstream_module.html](https://nginx.org/en/docs/http/ngx_http_upstream_module.html)

backup

- marks the server as a backup server. It will be passed requests when the primary servers are unavailable.

Can you please consider implementing this in some form? Just because of this one functionality, we would have to use  NGINX instead of Traefik :(.

I'm very much cheering for this feature as well! The use case is multiple data centers, but I'd like not to send traffic to other regions unless servers in the local region is down.
Any update about this feature? I spent days reading the whole doc and understanding traefik to realize in the end that it was not possible to have such a basic but essential function. For now it seems round-robin only :/
I'm looking for the same feature
Any idea if/when this will be available ?
I wanted to implement blue/green deployments with resilience on a nomad cluster with consul and traefik, I hope my example helps someone. 

I have a nomad cluster with blue/green instances in tomcats/docker and consul for service discovery that is used by traefik. Green instances register themselves in consul with tags 
traefik.http.routers.website-green.priority=123
traefik.http.routers.website-green.rule=Host("website") || Host("green.website")

similar for blue instance:

traefik.http.routers.website-blue.priority=120
traefik.http.routers.website-blue.rule=Host("website") || Host("blue.website")

This way traefik routes to green instance by default. In case the green instance dies, traefik will route to the blue instance. This mechanism can also be used for blue/green deployments in general: you deploy the blue instance, check it by accessing blue.website and if it's ok, promote it by increasing blue priority to 126 and later deploy green/increase green priority to 124.
Sounds great. I will try 👍
How about changing the `Retry` middleware to retry not the same backend but from a different backend? This would also finally enable us to do zero downtime deployments while using Docker service discovery. 
> I wanted to implement blue/green deployments with resilience on a nomad cluster with consul and traefik, I hope my example helps someone.
> 
> I have a nomad cluster with blue/green instances in tomcats/docker and consul for service discovery that is used by traefik. Green instances register themselves in consul with tags traefik.http.routers.website-green.priority=123 traefik.http.routers.website-green.rule=Host("website") || Host("green.website")
> 
> similar for blue instance:
> 
> traefik.http.routers.website-blue.priority=120 traefik.http.routers.website-blue.rule=Host("website") || Host("blue.website")
> 
> This way traefik routes to green instance by default. In case the green instance dies, traefik will route to the blue instance. This mechanism can also be used for blue/green deployments in general: you deploy the blue instance, check it by accessing blue.website and if it's ok, promote it by increasing blue priority to 126 and later deploy green/increase green priority to 124.

@jazzmuesli this seems to work great, thanks for sharing ❤️ . In addition to the `router` I had to also "namespace" the `service`, otherwise Traefik gets confused about differences in configuration of the same service between different backends.

Almost 5 years and no love from the Traefik team. 😕
The workaround from @jazzmuesli does work though
@mannharleen @jazzmuesli It didn't end up working for me after all because backends were added before they were healthy, see https://github.com/traefik/traefik/issues/8570

Fix patch:
diff --git a/docs/content/reference/dynamic-configuration/file.toml b/docs/content/reference/dynamic-configuration/file.toml
index 0aad383d69..08c45f0b70 100644
--- a/docs/content/reference/dynamic-configuration/file.toml
+++ b/docs/content/reference/dynamic-configuration/file.toml
@@ -95,6 +95,12 @@
             secure = true
             httpOnly = true
             sameSite = "foobar"
+    [http.services.Service04]
+      [http.services.Service04.failover]
+        service = "foobar"
+        fallback = "foobar"
+
+      [http.services.Service04.failover.healthCheck]
   [http.middlewares]
     [http.middlewares.Middleware00]
       [http.middlewares.Middleware00.addPrefix]
diff --git a/docs/content/reference/dynamic-configuration/file.yaml b/docs/content/reference/dynamic-configuration/file.yaml
index 846e9ea2c3..0fa87fa6cc 100644
--- a/docs/content/reference/dynamic-configuration/file.yaml
+++ b/docs/content/reference/dynamic-configuration/file.yaml
@@ -95,6 +95,11 @@ http:
             secure: true
             httpOnly: true
             sameSite: foobar
+    Service04:
+      failover:
+        service: foobar
+        fallback: foobar
+        healthCheck: {}
   middlewares:
     Middleware00:
       addPrefix:
diff --git a/docs/content/reference/dynamic-configuration/kv-ref.md b/docs/content/reference/dynamic-configuration/kv-ref.md
index 0df01e697c..591052b6ba 100644
--- a/docs/content/reference/dynamic-configuration/kv-ref.md
+++ b/docs/content/reference/dynamic-configuration/kv-ref.md
@@ -228,6 +228,9 @@
 | `traefik/http/services/Service03/weighted/sticky/cookie/name` | `foobar` |
 | `traefik/http/services/Service03/weighted/sticky/cookie/sameSite` | `foobar` |
 | `traefik/http/services/Service03/weighted/sticky/cookie/secure` | `true` |
+| `traefik/http/services/Service04/failover/fallback` | `foobar` |
+| `traefik/http/services/Service04/failover/healthCheck` | `` |
+| `traefik/http/services/Service04/failover/service` | `foobar` |
 | `traefik/tcp/middlewares/Middleware00/ipWhiteList/sourceRange/0` | `foobar` |
 | `traefik/tcp/middlewares/Middleware00/ipWhiteList/sourceRange/1` | `foobar` |
 | `traefik/tcp/routers/TCPRouter0/entryPoints/0` | `foobar` |
diff --git a/docs/content/routing/services/index.md b/docs/content/routing/services/index.md
index 6b3d35701b..7205177f90 100644
--- a/docs/content/routing/services/index.md
+++ b/docs/content/routing/services/index.md
@@ -1212,6 +1212,139 @@ http:
         url = "http://private-ip-server-2/"
 ```
 
+### Failover (service)
+
+A failover service job is to forward all requests to a fallback service when the main service becomes unreachable.
+
+!!! info "Relation to HealthCheck"
+
+    The failover service relies on the HealthCheck system to get notified when its main service becomes unreachable,
+    which means HealthCheck needs to be enabled and functional on the main service.
+    However, HealthCheck does not need to be enabled on the failover service itself for it to be functional.
+    It is only required in order to propagate upwards the information when the failover itself becomes down
+    (i.e. both its main and its fallback are down too).
+
+!!! info "Supported Providers"
+
+    This strategy can currently only be defined with the [File](../../providers/file.md) provider.
+
+```yaml tab="YAML"
+## Dynamic configuration
+http:
+  services:
+    app:
+      failover:
+        service: main
+        fallback: backup
+
+    main:
+      loadBalancer:
+        healthCheck:
+          path: /status
+          interval: 10s
+          timeout: 3s
+        servers:
+        - url: "http://private-ip-server-1/"
+
+    backup:
+      loadBalancer:
+        servers:
+        - url: "http://private-ip-server-2/"
+```
+
+```toml tab="TOML"
+## Dynamic configuration
+[http.services]
+  [http.services.app]
+    [http.services.app.failover]
+      service = "main"
+      fallback = "backup"
+
+  [http.services.main]
+    [http.services.main.loadBalancer]
+      [http.services.main.loadBalancer.healthCheck]
+        path = "/health"
+        interval = "10s"
+        timeout = "3s"
+      [[http.services.main.loadBalancer.servers]]
+        url = "http://private-ip-server-1/"
+
+  [http.services.backup]
+    [http.services.backup.loadBalancer]
+      [[http.services.backup.loadBalancer.servers]]
+        url = "http://private-ip-server-2/"
+```
+
+#### Health Check
+
+HealthCheck enables automatic self-healthcheck for this service,
+i.e. if the main and the fallback services become unreachable,
+the information is propagated upwards to its parent.
+
+!!! info "All or nothing"
+
+    If HealthCheck is enabled for a given service, but any of its descendants does
+    not have it enabled, the creation of the service will fail.
+
+    HealthCheck on a Failover service can currently only be defined with the [File](../../providers/file.md) provider.
+
+```yaml tab="YAML"
+## Dynamic configuration
+http:
+  services:
+    app:
+      failover:
+        healthCheck: {}
+        service: main
+        fallback: backup
+
+    main:
+      loadBalancer:
+        healthCheck:
+          path: /status
+          interval: 10s
+          timeout: 3s
+        servers:
+        - url: "http://private-ip-server-1/"
+
+    backup:
+      loadBalancer:
+        healthCheck:
+          path: /status
+          interval: 10s
+          timeout: 3s
+        servers:
+        - url: "http://private-ip-server-2/"
+```
+
+```toml tab="TOML"
+## Dynamic configuration
+[http.services]
+  [http.services.app]
+    [http.services.app.failover.healthCheck]
+    [http.services.app.failover]
+      service = "main"
+      fallback = "backup"
+
+  [http.services.main]
+    [http.services.main.loadBalancer]
+      [http.services.main.loadBalancer.healthCheck]
+        path = "/health"
+        interval = "10s"
+        timeout = "3s"
+      [[http.services.main.loadBalancer.servers]]
+        url = "http://private-ip-server-1/"
+
+  [http.services.backup]
+    [http.services.backup.loadBalancer]
+      [http.services.backup.loadBalancer.healthCheck]
+        path = "/health"
+        interval = "10s"
+        timeout = "3s"
+      [[http.services.backup.loadBalancer.servers]]
+        url = "http://private-ip-server-2/"
+```
+
 ## Configuring TCP Services
 
 ### General
diff --git a/pkg/config/dynamic/http_config.go b/pkg/config/dynamic/http_config.go
index ea4e979640..014cfe2058 100644
--- a/pkg/config/dynamic/http_config.go
+++ b/pkg/config/dynamic/http_config.go
@@ -35,6 +35,7 @@ type Service struct {
 	LoadBalancer *ServersLoadBalancer `json:"loadBalancer,omitempty" toml:"loadBalancer,omitempty" yaml:"loadBalancer,omitempty" export:"true"`
 	Weighted     *WeightedRoundRobin  `json:"weighted,omitempty" toml:"weighted,omitempty" yaml:"weighted,omitempty" label:"-" export:"true"`
 	Mirroring    *Mirroring           `json:"mirroring,omitempty" toml:"mirroring,omitempty" yaml:"mirroring,omitempty" label:"-" export:"true"`
+	Failover     *Failover            `json:"failover,omitempty" toml:"failover,omitempty" yaml:"failover,omitempty" label:"-" export:"true"`
 }
 
 // +k8s:deepcopy-gen=true
@@ -76,6 +77,15 @@ func (m *Mirroring) SetDefaults() {
 
 // +k8s:deepcopy-gen=true
 
+// Failover holds the Failover configuration.
+type Failover struct {
+	Service     string       `json:"service,omitempty" toml:"service,omitempty" yaml:"service,omitempty" export:"true"`
+	Fallback    string       `json:"fallback,omitempty" toml:"fallback,omitempty" yaml:"fallback,omitempty" export:"true"`
+	HealthCheck *HealthCheck `json:"healthCheck,omitempty" toml:"healthCheck,omitempty" yaml:"healthCheck,omitempty" label:"allowEmpty" file:"allowEmpty" export:"true"`
+}
+
+// +k8s:deepcopy-gen=true
+
 // MirrorService holds the MirrorService configuration.
 type MirrorService struct {
 	Name    string `json:"name,omitempty" toml:"name,omitempty" yaml:"name,omitempty" export:"true"`
@@ -98,7 +108,7 @@ type WeightedRoundRobin struct {
 
 // +k8s:deepcopy-gen=true
 
-// WRRService is a reference to a service load-balanced with weighted round robin.
+// WRRService is a reference to a service load-balanced with weighted round-robin.
 type WRRService struct {
 	Name   string `json:"name,omitempty" toml:"name,omitempty" yaml:"name,omitempty" export:"true"`
 	Weight *int   `json:"weight,omitempty" toml:"weight,omitempty" yaml:"weight,omitempty" export:"true"`
diff --git a/pkg/config/dynamic/zz_generated.deepcopy.go b/pkg/config/dynamic/zz_generated.deepcopy.go
index be07061df9..5870481802 100644
--- a/pkg/config/dynamic/zz_generated.deepcopy.go
+++ b/pkg/config/dynamic/zz_generated.deepcopy.go
@@ -285,6 +285,27 @@ func (in *ErrorPage) DeepCopy() *ErrorPage {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *Failover) DeepCopyInto(out *Failover) {
+	*out = *in
+	if in.HealthCheck != nil {
+		in, out := &in.HealthCheck, &out.HealthCheck
+		*out = new(HealthCheck)
+		**out = **in
+	}
+	return
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Failover.
+func (in *Failover) DeepCopy() *Failover {
+	if in == nil {
+		return nil
+	}
+	out := new(Failover)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *ForwardAuth) DeepCopyInto(out *ForwardAuth) {
 	*out = *in
@@ -1171,6 +1192,11 @@ func (in *Service) DeepCopyInto(out *Service) {
 		*out = new(Mirroring)
 		(*in).DeepCopyInto(*out)
 	}
+	if in.Failover != nil {
+		in, out := &in.Failover, &out.Failover
+		*out = new(Failover)
+		(*in).DeepCopyInto(*out)
+	}
 	return
 }
 
diff --git a/pkg/server/service/loadbalancer/failover/failover.go b/pkg/server/service/loadbalancer/failover/failover.go
new file mode 100644
index 0000000000..06d9fecd78
--- /dev/null
+++ b/pkg/server/service/loadbalancer/failover/failover.go
@@ -0,0 +1,140 @@
+package failover
+
+import (
+	"context"
+	"errors"
+	"net/http"
+	"sync"
+
+	"github.com/traefik/traefik/v2/pkg/config/dynamic"
+	"github.com/traefik/traefik/v2/pkg/log"
+)
+
+// Failover is an http.Handler that can forward requests to the fallback handler
+// when the main handler status is down.
+type Failover struct {
+	wantsHealthCheck bool
+	handler          http.Handler
+	fallbackHandler  http.Handler
+	// updaters is the list of hooks that are run (to update the Failover
+	// parent(s)), whenever the Failover status changes.
+	updaters []func(bool)
+
+	handlerStatusMu sync.RWMutex
+	handlerStatus   bool
+
+	fallbackStatusMu sync.RWMutex
+	fallbackStatus   bool
+}
+
+// New creates a new Failover handler.
+func New(hc *dynamic.HealthCheck) *Failover {
+	return &Failover{
+		wantsHealthCheck: hc != nil,
+	}
+}
+
+// RegisterStatusUpdater adds fn to the list of hooks that are run when the
+// status of the Failover changes.
+// Not thread safe.
+func (f *Failover) RegisterStatusUpdater(fn func(up bool)) error {
+	if !f.wantsHealthCheck {
+		return errors.New("healthCheck not enabled in config for this failover service")
+	}
+
+	f.updaters = append(f.updaters, fn)
+
+	return nil
+}
+
+func (f *Failover) ServeHTTP(w http.ResponseWriter, req *http.Request) {
+	f.handlerStatusMu.RLock()
+	handlerStatus := f.handlerStatus
+	f.handlerStatusMu.RUnlock()
+
+	if handlerStatus {
+		f.handler.ServeHTTP(w, req)
+		return
+	}
+
+	f.fallbackStatusMu.RLock()
+	fallbackStatus := f.fallbackStatus
+	f.fallbackStatusMu.RUnlock()
+
+	if fallbackStatus {
+		f.fallbackHandler.ServeHTTP(w, req)
+		return
+	}
+
+	http.Error(w, http.StatusText(http.StatusServiceUnavailable), http.StatusServiceUnavailable)
+}
+
+// SetHandler sets the main http.Handler.
+func (f *Failover) SetHandler(handler http.Handler) {
+	f.handlerStatusMu.Lock()
+	defer f.handlerStatusMu.Unlock()
+
+	f.handler = handler
+	f.handlerStatus = true
+}
+
+// SetHandlerStatus sets the main handler status.
+func (f *Failover) SetHandlerStatus(ctx context.Context, up bool) {
+	f.handlerStatusMu.Lock()
+	defer f.handlerStatusMu.Unlock()
+
+	status := "DOWN"
+	if up {
+		status = "UP"
+	}
+
+	if up == f.handlerStatus {
+		// We're still with the same status, no need to propagate.
+		log.FromContext(ctx).Debugf("Still %s, no need to propagate", status)
+		return
+	}
+
+	log.FromContext(ctx).Debugf("Propagating new %s status", status)
+	f.handlerStatus = up
+
+	for _, fn := range f.updaters {
+		// Failover service status is set to DOWN
+		// when main and fallback handlers have a DOWN status.
+		fn(f.handlerStatus || f.fallbackStatus)
+	}
+}
+
+// SetFallbackHandler sets the fallback http.Handler.
+func (f *Failover) SetFallbackHandler(handler http.Handler) {
+	f.fallbackStatusMu.Lock()
+	defer f.fallbackStatusMu.Unlock()
+
+	f.fallbackHandler = handler
+	f.fallbackStatus = true
+}
+
+// SetFallbackHandlerStatus sets the fallback handler status.
+func (f *Failover) SetFallbackHandlerStatus(ctx context.Context, up bool) {
+	f.fallbackStatusMu.Lock()
+	defer f.fallbackStatusMu.Unlock()
+
+	status := "DOWN"
+	if up {
+		status = "UP"
+	}
+
+	if up == f.fallbackStatus {
+		// We're still with the same status, no need to propagate.
+		log.FromContext(ctx).Debugf("Still %s, no need to propagate", status)
+		return
+	}
+
+	log.FromContext(ctx).Debugf("Propagating new %s status", status)
+	f.fallbackStatus = up
+
+	for _, fn := range f.updaters {
+		// Failover service status is set to DOWN
+		// when main and fallback handlers have a DOWN status.
+		fn(f.handlerStatus || f.fallbackStatus)
+	}
+}
diff --git a/pkg/server/service/loadbalancer/wrr/wrr.go b/pkg/server/service/loadbalancer/wrr/wrr.go
index a1fa085aa7..d7261d95b6 100644
--- a/pkg/server/service/loadbalancer/wrr/wrr.go
+++ b/pkg/server/service/loadbalancer/wrr/wrr.go
@@ -29,7 +29,7 @@ type stickyCookie struct {
 // (https://en.wikipedia.org/wiki/Earliest_deadline_first_scheduling)
 // Each pick from the schedule has the earliest deadline entry selected.
 // Entries have deadlines set at currentDeadline + 1 / weight,
-// providing weighted round robin behavior with floating point weights and an O(log n) pick time.
+// providing weighted round-robin behavior with floating point weights and an O(log n) pick time.
 type Balancer struct {
 	stickyCookie     *stickyCookie
 	wantsHealthCheck bool
@@ -230,6 +230,7 @@ func (b *Balancer) AddService(name string, handler http.Handler, weight *int) {
 	if weight != nil {
 		w = *weight
 	}
+
 	if w <= 0 { // non-positive weight is meaningless
 		return
 	}
diff --git a/pkg/server/service/service.go b/pkg/server/service/service.go
index 93e6f23b5f..e2f8785949 100644
--- a/pkg/server/service/service.go
+++ b/pkg/server/service/service.go
@@ -23,6 +23,7 @@ import (
 	"github.com/traefik/traefik/v2/pkg/safe"
 	"github.com/traefik/traefik/v2/pkg/server/cookie"
 	"github.com/traefik/traefik/v2/pkg/server/provider"
+	"github.com/traefik/traefik/v2/pkg/server/service/loadbalancer/failover"
 	"github.com/traefik/traefik/v2/pkg/server/service/loadbalancer/mirror"
 	"github.com/traefik/traefik/v2/pkg/server/service/loadbalancer/wrr"
 	"github.com/vulcand/oxy/roundrobin"
@@ -116,6 +117,13 @@ func (m *Manager) BuildHTTP(rootCtx context.Context, serviceName string) (http.H
 			conf.AddError(err, true)
 			return nil, err
 		}
+	case conf.Failover != nil:
+		var err error
+		lb, err = m.getFailoverServiceHandler(ctx, serviceName, conf.Failover)
+		if err != nil {
+			conf.AddError(err, true)
+			return nil, err
+		}
 	default:
 		sErr := fmt.Errorf("the service %q does not have any type defined", serviceName)
 		conf.AddError(sErr, true)
@@ -125,6 +133,53 @@ func (m *Manager) BuildHTTP(rootCtx context.Context, serviceName string) (http.H
 	return lb, nil
 }
 
+func (m *Manager) getFailoverServiceHandler(ctx context.Context, serviceName string, config *dynamic.Failover) (http.Handler, error) {
+	f := failover.New(config.HealthCheck)
+
+	serviceHandler, err := m.BuildHTTP(ctx, config.Service)
+	if err != nil {
+		return nil, err
+	}
+
+	f.SetHandler(serviceHandler)
+
+	updater, ok := serviceHandler.(healthcheck.StatusUpdater)
+	if !ok {
+		return nil, fmt.Errorf("child service %v of %v not a healthcheck.StatusUpdater (%T)", config.Service, serviceName, serviceHandler)
+	}
+
+	if err := updater.RegisterStatusUpdater(func(up bool) {
+		f.SetHandlerStatus(ctx, up)
+	}); err != nil {
+		return nil, fmt.Errorf("cannot register %v as updater for %v: %w", config.Service, serviceName, err)
+	}
+
+	fallbackHandler, err := m.BuildHTTP(ctx, config.Fallback)
+	if err != nil {
+		return nil, err
+	}
+
+	f.SetFallbackHandler(fallbackHandler)
+
+	// Do not report the health of the fallback handler.
+	if config.HealthCheck == nil {
+		return f, nil
+	}
+
+	fallbackUpdater, ok := fallbackHandler.(healthcheck.StatusUpdater)
+	if !ok {
+		return nil, fmt.Errorf("child service %v of %v not a healthcheck.StatusUpdater (%T)", config.Fallback, serviceName, fallbackHandler)
+	}
+
+	if err := fallbackUpdater.RegisterStatusUpdater(func(up bool) {
+		f.SetFallbackHandlerStatus(ctx, up)
+	}); err != nil {
+		return nil, fmt.Errorf("cannot register %v as updater for %v: %w", config.Fallback, serviceName, err)
+	}
+
+	return f, nil
+}
+
 func (m *Manager) getMirrorServiceHandler(ctx context.Context, config *dynamic.Mirroring) (http.Handler, error) {
 	serviceHandler, err := m.BuildHTTP(ctx, config.Service)
 	if err != nil {
@@ -164,6 +219,7 @@ func (m *Manager) getWRRServiceHandler(ctx context.Context, serviceName string,
 		}
 
 		balancer.AddService(service.Name, serviceHandler, service.Weight)
+
 		if config.HealthCheck == nil {
 			continue
 		}
diff --git a/webui/src/components/_commons/PanelServiceDetails.vue b/webui/src/components/_commons/PanelServiceDetails.vue
index a6052935bd..71a3c54dd1 100644
--- a/webui/src/components/_commons/PanelServiceDetails.vue
+++ b/webui/src/components/_commons/PanelServiceDetails.vue
@@ -39,8 +39,9 @@
             <div class="text-subtitle2">Main Service</div>
             <q-chip
               dense
-              class="app-chip app-chip-name">
+              class="app-chip app-chip-name app-chip-overflow">
               {{ data.mirroring.service }}
+              <q-tooltip>{{ data.mirroring.service }}</q-tooltip>
             </q-chip>
           </div>
         </div>
@@ -80,6 +81,34 @@
         </div>
       </q-card-section>
 
+      <q-card-section v-if="data.failover && data.failover.service">
+        <div class="row items-start no-wrap">
+          <div class="col">
+            <div class="text-subtitle2">Main Service</div>
+            <q-chip
+              dense
+              class="app-chip app-chip-name app-chip-overflow">
+              {{ data.failover.service }}
+              <q-tooltip>{{ data.failover.service }}</q-tooltip>
+            </q-chip>
+          </div>
+        </div>
+      </q-card-section>
+
+      <q-card-section v-if="data.failover && data.failover.fallback">
+        <div class="row items-start no-wrap">
+          <div class="col">
+            <div class="text-subtitle2">Fallback Service</div>
+            <q-chip
+              dense
+              class="app-chip app-chip-name app-chip-overflow">
+              {{ data.failover.fallback }}
+              <q-tooltip>{{ data.failover.fallback }}</q-tooltip>
+            </q-chip>
+          </div>
+        </div>
+      </q-card-section>
+
       <q-separator v-if="sticky" />
       <StickyServiceDetails v-if="sticky" :sticky="sticky" :dense="dense"/>
     </q-scroll-area>


Test patch:
diff --git a/pkg/provider/kv/kv_test.go b/pkg/provider/kv/kv_test.go
index ba623f4a34..b41fada2ab 100644
--- a/pkg/provider/kv/kv_test.go
+++ b/pkg/provider/kv/kv_test.go
@@ -69,6 +69,8 @@ func Test_buildConfiguration(t *testing.T) {
 		"traefik/http/services/Service03/weighted/services/0/weight":                                 "42",
 		"traefik/http/services/Service03/weighted/services/1/name":                                   "foobar",
 		"traefik/http/services/Service03/weighted/services/1/weight":                                 "42",
+		"traefik/http/services/Service04/failover/service":                                           "foobar",
+		"traefik/http/services/Service04/failover/fallback":                                          "foobar",
 		"traefik/http/middlewares/Middleware08/forwardAuth/authResponseHeaders/0":                    "foobar",
 		"traefik/http/middlewares/Middleware08/forwardAuth/authResponseHeaders/1":                    "foobar",
 		"traefik/http/middlewares/Middleware08/forwardAuth/authRequestHeaders/0":                     "foobar",
@@ -688,6 +690,12 @@ func Test_buildConfiguration(t *testing.T) {
 						},
 					},
 				},
+				"Service04": {
+					Failover: &dynamic.Failover{
+						Service:  "foobar",
+						Fallback: "foobar",
+					},
+				},
 			},
 		},
 		TCP: &dynamic.TCPConfiguration{
diff --git a/pkg/server/service/loadbalancer/failover/failover_test.go b/pkg/server/service/loadbalancer/failover/failover_test.go
new file mode 100644
index 0000000000..9f8d382152
--- /dev/null
+++ b/pkg/server/service/loadbalancer/failover/failover_test.go
@@ -0,0 +1,163 @@
+package failover
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"github.com/traefik/traefik/v2/pkg/config/dynamic"
+)
+
+type responseRecorder struct {
+	*httptest.ResponseRecorder
+	save     map[string]int
+	sequence []string
+	status   []int
+}
+
+func (r *responseRecorder) WriteHeader(statusCode int) {
+	r.save[r.Header().Get("server")]++
+	r.sequence = append(r.sequence, r.Header().Get("server"))
+	r.status = append(r.status, statusCode)
+	r.ResponseRecorder.WriteHeader(statusCode)
+}
+
+func TestFailover(t *testing.T) {
+	failover := New(&dynamic.HealthCheck{})
+
+	status := true
+	require.NoError(t, failover.RegisterStatusUpdater(func(up bool) {
+		status = up
+	}))
+
+	failover.SetHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "handler")
+		rw.WriteHeader(http.StatusOK)
+	}))
+
+	failover.SetFallbackHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "fallback")
+		rw.WriteHeader(http.StatusOK)
+	}))
+
+	recorder := &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 1, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, []int{200}, recorder.status)
+	assert.True(t, status)
+
+	failover.SetHandlerStatus(context.Background(), false)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 0, recorder.save["handler"])
+	assert.Equal(t, 1, recorder.save["fallback"])
+	assert.Equal(t, []int{200}, recorder.status)
+	assert.True(t, status)
+
+	failover.SetFallbackHandlerStatus(context.Background(), false)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 0, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, []int{503}, recorder.status)
+	assert.False(t, status)
+}
+
+func TestFailoverDownThenUp(t *testing.T) {
+	failover := New(nil)
+
+	failover.SetHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "handler")
+		rw.WriteHeader(http.StatusOK)
+	}))
+
+	failover.SetFallbackHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "fallback")
+		rw.WriteHeader(http.StatusOK)
+	}))
+
+	recorder := &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 1, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, []int{200}, recorder.status)
+
+	failover.SetHandlerStatus(context.Background(), false)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 0, recorder.save["handler"])
+	assert.Equal(t, 1, recorder.save["fallback"])
+	assert.Equal(t, []int{200}, recorder.status)
+
+	failover.SetHandlerStatus(context.Background(), true)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	failover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 1, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, []int{200}, recorder.status)
+}
+
+func TestFailoverPropagate(t *testing.T) {
+	failover := New(&dynamic.HealthCheck{})
+	failover.SetHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "handler")
+		rw.WriteHeader(http.StatusOK)
+	}))
+	failover.SetFallbackHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "fallback")
+		rw.WriteHeader(http.StatusOK)
+	}))
+
+	topFailover := New(nil)
+	topFailover.SetHandler(failover)
+	topFailover.SetFallbackHandler(http.HandlerFunc(func(rw http.ResponseWriter, req *http.Request) {
+		rw.Header().Set("server", "topFailover")
+		rw.WriteHeader(http.StatusOK)
+	}))
+	err := failover.RegisterStatusUpdater(func(up bool) {
+		topFailover.SetHandlerStatus(context.Background(), up)
+	})
+	require.NoError(t, err)
+
+	recorder := &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	topFailover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 1, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, 0, recorder.save["topFailover"])
+	assert.Equal(t, []int{200}, recorder.status)
+
+	failover.SetHandlerStatus(context.Background(), false)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	topFailover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 0, recorder.save["handler"])
+	assert.Equal(t, 1, recorder.save["fallback"])
+	assert.Equal(t, 0, recorder.save["topFailover"])
+	assert.Equal(t, []int{200}, recorder.status)
+
+	failover.SetFallbackHandlerStatus(context.Background(), false)
+
+	recorder = &responseRecorder{ResponseRecorder: httptest.NewRecorder(), save: map[string]int{}}
+	topFailover.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+
+	assert.Equal(t, 0, recorder.save["handler"])
+	assert.Equal(t, 0, recorder.save["fallback"])
+	assert.Equal(t, 1, recorder.save["topFailover"])
+	assert.Equal(t, []int{200}, recorder.status)
+}