Repository: scylladb/scylla-operator
Pull request #605 (2021-05-24 11:40:53)

Provision autogenerated auth token for Scylla API 
From now on Agent requires a proper authorization token for requests.
Token is provisioned by the Operator as a Secret for each ScyllaCluster.
If user already enabled authentication by filling it in optional Agent
config Secret, its value will be used, if not, a random autogenerated string will be saved.


Patch:
diff --git a/deploy/operator/00_clusterrole_def.yaml b/deploy/operator/00_clusterrole_def.yaml
index d72e885fe..65e1a2946 100644
--- a/deploy/operator/00_clusterrole_def.yaml
+++ b/deploy/operator/00_clusterrole_def.yaml
@@ -58,6 +58,8 @@ rules:
   - get
   - list
   - watch
+  - create
+  - update
 - apiGroups:
   - ""
   resources:
diff --git a/deploy/operator/00_scyllacluster_member_clusterrole_def.yaml b/deploy/operator/00_scyllacluster_member_clusterrole_def.yaml
index 38b70662b..9276ea4c6 100644
--- a/deploy/operator/00_scyllacluster_member_clusterrole_def.yaml
+++ b/deploy/operator/00_scyllacluster_member_clusterrole_def.yaml
@@ -11,6 +11,12 @@ rules:
   - pods
   verbs:
   - get
+- apiGroups:
+  - ""
+  resources:
+  - secrets
+  verbs:
+  - get
 - apiGroups:
   - ""
   resources:
diff --git a/docs/source/generic.md b/docs/source/generic.md
index fe34dd136..31d26f198 100644
--- a/docs/source/generic.md
+++ b/docs/source/generic.md
@@ -249,8 +249,8 @@ The operator will then apply the overridable properties `prefer_local` and `dc_s
 ## Configure Scylla Manager Agent
 
 The operator creates a second container for each scylla instance that runs [Scylla Manager Agent](https://hub.docker.com/r/scylladb/scylla-manager-agent).
-This container serves as a sidecar and it's the main endpoint for [Scylla Manager](https://hub.docker.com/r/scylladb/scylla-manager) when interacting with Scylla.
-The Scylla Manager Agent can be configured with various things such as the security token used to allow access to it's API.
+This container serves as a sidecar and it's the main endpoint for interacting with Scylla API.
+The Scylla Manager Agent can be configured with various things such as the security token used to allow access to Scylla API and storage providers for backups.
 
 To configure the agent you just create a new secret called _scylla-agent-config-secret_ and populate it with the contents in the `scylla-manager-agent.yaml` file like this:
 ```console
@@ -259,18 +259,13 @@ kubectl create secret -n scylla generic scylla-agent-config-secret --from-file s
 
 See [Scylla Manager Agent configuration](https://docs.scylladb.com/operating-scylla/manager/2.0/agent-configuration-file/) for a complete reference of the Scylla Manager agent config file.
 
-In order for the operator to be able to use the agent it may need to be configured accordingly. For example it needs a matching security token.
-The operator uses a file called `scylla-client.yaml` for this and the content is today limited to two properties:
-```yaml
-auth_token: the_token
-```
-To configure the operator you just create a new config-map called _scylla-client-config-secret_ and populate it with the contents in the `scylla-client.yaml` file like this:
-```console
-kubectl create secret -n scylla generic scylla-client-config-secret --from-file scylla-client.yaml
-```
-After a restart the operator will use the security token when it interacts with scylla via the agent.
+### Scylla Manager Agent auth token
+
+Operator provisions Agent auth token by copying value from user provided config secret or auto generates it if it's empty.
+To check which value is being used, decode content of `<cluster-name>-auth-token` secret. 
+To change it simply remove the secret. Operator will create a new one. To pick up the change in the cluster, initiate a rolling restart.
 
- ### Set up monitoring
+## Set up monitoring
 
 To set up monitoring using Prometheus and Grafana follow [this guide](monitoring.md).
 
diff --git a/examples/common/operator.yaml b/examples/common/operator.yaml
index de6e64490..9c5013ea4 100644
--- a/examples/common/operator.yaml
+++ b/examples/common/operator.yaml
@@ -69,6 +69,8 @@ rules:
   - get
   - list
   - watch
+  - create
+  - update
 - apiGroups:
   - ""
   resources:
@@ -2030,6 +2032,12 @@ rules:
   - pods
   verbs:
   - get
+- apiGroups:
+  - ""
+  resources:
+  - secrets
+  verbs:
+  - get
 - apiGroups:
   - ""
   resources:
diff --git a/helm/scylla-operator/templates/clusterrole_def.yaml b/helm/scylla-operator/templates/clusterrole_def.yaml
index d72e885fe..65e1a2946 100644
--- a/helm/scylla-operator/templates/clusterrole_def.yaml
+++ b/helm/scylla-operator/templates/clusterrole_def.yaml
@@ -58,6 +58,8 @@ rules:
   - get
   - list
   - watch
+  - create
+  - update
 - apiGroups:
   - ""
   resources:
diff --git a/helm/scylla-operator/templates/scyllacluster_member_clusterrole_def.yaml b/helm/scylla-operator/templates/scyllacluster_member_clusterrole_def.yaml
index 38b70662b..9276ea4c6 100644
--- a/helm/scylla-operator/templates/scyllacluster_member_clusterrole_def.yaml
+++ b/helm/scylla-operator/templates/scyllacluster_member_clusterrole_def.yaml
@@ -11,6 +11,12 @@ rules:
   - pods
   verbs:
   - get
+- apiGroups:
+  - ""
+  resources:
+  - secrets
+  verbs:
+  - get
 - apiGroups:
   - ""
   resources:
diff --git a/pkg/controllers/cluster/actions/upgrade_version.go b/pkg/controllers/cluster/actions/upgrade_version.go
index 0fa12359a..7fb1572ce 100644
--- a/pkg/controllers/cluster/actions/upgrade_version.go
+++ b/pkg/controllers/cluster/actions/upgrade_version.go
@@ -14,6 +14,7 @@ import (
 	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
 	"github.com/scylladb/scylla-operator/pkg/controllers/cluster/resource"
 	"github.com/scylladb/scylla-operator/pkg/controllers/cluster/util"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
 	"github.com/scylladb/scylla-operator/pkg/naming"
 	"github.com/scylladb/scylla-operator/pkg/scyllaclient"
 	"github.com/scylladb/scylla-operator/pkg/util/fsm"
@@ -25,6 +26,7 @@ import (
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/util/wait"
 	"k8s.io/client-go/kubernetes"
+	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"k8s.io/client-go/tools/record"
 	"k8s.io/client-go/util/retry"
 	"sigs.k8s.io/controller-runtime/pkg/client"
@@ -63,8 +65,12 @@ func (a *ClusterVersionUpgrade) Name() string {
 	return ClusterVersionUpgradeAction
 }
 
-var ScyllaClientForClusterFunc = func(ctx context.Context, cc client.Client, hosts []string, logger log.Logger) (*scyllaclient.Client, error) {
-	cfg := scyllaclient.DefaultConfig(hosts...)
+var ScyllaClientForClusterFunc = func(ctx context.Context, client corev1client.CoreV1Interface, cluster *scyllav1.ScyllaCluster, hosts []string, logger log.Logger) (*scyllaclient.Client, error) {
+	authToken, err := helpers.GetAgentAuthToken(ctx, client, cluster.Name, cluster.Namespace)
+	if err != nil {
+		return nil, fmt.Errorf("get auth token: %w", err)
+	}
+	cfg := scyllaclient.DefaultConfig(authToken, hosts...)
 	return scyllaclient.NewClient(cfg, logger)
 }
 
@@ -153,7 +159,7 @@ func (a *ClusterVersionUpgrade) genericUpgrade(ctx context.Context) error {
 		return errors.Wrap(err, "host discovery")
 	}
 
-	a.ScyllaClient, err = ScyllaClientForClusterFunc(ctx, a.cc, hosts, a.logger)
+	a.ScyllaClient, err = ScyllaClientForClusterFunc(ctx, a.kubeClient.CoreV1(), a.Cluster, hosts, a.logger)
 	if err != nil {
 		return errors.Wrap(err, "create scylla client")
 	}
diff --git a/pkg/controllers/cluster/agent.go b/pkg/controllers/cluster/agent.go
new file mode 100644
index 000000000..a4e5a8f9c
--- /dev/null
+++ b/pkg/controllers/cluster/agent.go
@@ -0,0 +1,89 @@
+// Copyright (C) 2021 ScyllaDB
+
+package cluster
+
+import (
+	"context"
+	"fmt"
+
+	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
+	"github.com/scylladb/scylla-operator/pkg/controllers/cluster/resource"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
+	"github.com/scylladb/scylla-operator/pkg/naming"
+	"github.com/scylladb/scylla-operator/pkg/util/resourceapply"
+	"gopkg.in/yaml.v2"
+	apierrors "k8s.io/apimachinery/pkg/api/errors"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/util/rand"
+	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
+)
+
+type agentConfig struct {
+	AuthToken string `yaml:"auth_token"`
+}
+
+func (cc *ClusterReconciler) syncAgentAuthToken(ctx context.Context, cluster *scyllav1.ScyllaCluster) error {
+	secretName := naming.AgentAuthTokenSecretName(cluster.Name)
+	var token string
+
+	existing, err := cc.KubeClient.CoreV1().Secrets(cluster.Namespace).Get(ctx, secretName, metav1.GetOptions{})
+	if err != nil && !apierrors.IsNotFound(err) {
+		return fmt.Errorf("get auth token secret: %w", err)
+	}
+
+	if apierrors.IsNotFound(err) {
+		token = rand.String(128)
+	} else {
+		token, err = helpers.GetAgentAuthTokenFromSecret(existing)
+		if err != nil {
+			return fmt.Errorf("get token from secret: %w", err)
+		}
+	}
+
+	userToken, err := getAgentAuthTokenFromUserSecret(ctx, cc.KubeClient.CoreV1(), cluster)
+	if err != nil {
+		return fmt.Errorf("get auth token from user secret: %w", err)
+	}
+
+	// User token is preferred.
+	if userToken != "" {
+		token = userToken
+	}
+
+	secret, err := resource.MakeAgentAuthTokenSecret(cluster, token)
+	if err != nil {
+		return fmt.Errorf("make auth token secret: %w", err)
+	}
+
+	if err := resourceapply.ApplySecret(ctx, cc.Recorder, cc.Client, secret); err != nil {
+		return fmt.Errorf("apply secret: %w", err)
+	}
+
+	return nil
+}
+
+func getAgentAuthTokenFromUserSecret(ctx context.Context, client corev1client.CoreV1Interface, cluster *scyllav1.ScyllaCluster) (string, error) {
+	if len(cluster.Spec.Datacenter.Racks) == 0 {
+		return "", nil
+	}
+
+	agentSecretName := cluster.Spec.Datacenter.Racks[0].ScyllaAgentConfig
+	if agentSecretName == "" {
+		return "", nil
+	}
+
+	secret, err := client.Secrets(cluster.Namespace).Get(ctx, agentSecretName, metav1.GetOptions{})
+	if err != nil {
+		if apierrors.IsNotFound(err) {
+			return "", nil
+		}
+		return "", fmt.Errorf("can't get secret %q: %w", agentSecretName, err)
+	}
+
+	cfg := &agentConfig{}
+	if err := yaml.Unmarshal(secret.Data[naming.ScyllaAgentConfigFileName], &cfg); err != nil {
+		return "", fmt.Errorf("can't parse agent config: %w", err)
+	}
+
+	return cfg.AuthToken, nil
+}
diff --git a/pkg/controllers/cluster/cluster_controller.go b/pkg/controllers/cluster/cluster_controller.go
index a0be65a91..ad16b04c8 100644
--- a/pkg/controllers/cluster/cluster_controller.go
+++ b/pkg/controllers/cluster/cluster_controller.go
@@ -214,6 +214,19 @@ func (r *ClusterReconciler) SetupWithManager(mgr ctrl.Manager) error {
 		return errors.Wrap(err, "pod disruption budget watch setup")
 	}
 
+	// Watch Secrets created for ScyllaCluster
+	err = c.Watch(
+		&source.Kind{Type: &corev1.Secret{}},
+		&handler.EnqueueRequestForOwner{
+			IsController: true,
+			OwnerType:    &scyllav1.ScyllaCluster{},
+		},
+		predicate.ResourceVersionChangedPredicate{},
+	)
+	if err != nil {
+		return errors.Wrap(err, "secret watch setup")
+	}
+
 	return nil
 }
 
diff --git a/pkg/controllers/cluster/resource/resource.go b/pkg/controllers/cluster/resource/resource.go
index eb4418693..1f925bd85 100644
--- a/pkg/controllers/cluster/resource/resource.go
+++ b/pkg/controllers/cluster/resource/resource.go
@@ -7,7 +7,9 @@ import (
 
 	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
 	"github.com/scylladb/scylla-operator/pkg/controllers/cluster/util"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
 	"github.com/scylladb/scylla-operator/pkg/naming"
+	"gopkg.in/yaml.v2"
 	appsv1 "k8s.io/api/apps/v1"
 	corev1 "k8s.io/api/core/v1"
 	"k8s.io/api/policy/v1beta1"
@@ -17,6 +19,11 @@ import (
 	"k8s.io/utils/pointer"
 )
 
+const (
+	scyllaAgentConfigVolumeName    = "scylla-agent-config-volume"
+	scyllaAgentAuthTokenVolumeName = "scylla-agent-auth-token-volume"
+)
+
 func HeadlessServiceForCluster(c *scyllav1.ScyllaCluster) *corev1.Service {
 	return &corev1.Service{
 		ObjectMeta: metav1.ObjectMeta{
@@ -174,7 +181,7 @@ func StatefulSetForRack(r scyllav1.RackSpec, c *scyllav1.ScyllaCluster, sidecarI
 							},
 						},
 						{
-							Name: "scylla-agent-config-volume",
+							Name: scyllaAgentConfigVolumeName,
 							VolumeSource: corev1.VolumeSource{
 								Secret: &corev1.SecretVolumeSource{
 									SecretName: stringOrDefault(r.ScyllaAgentConfig, "scylla-agent-config-secret"),
@@ -191,6 +198,14 @@ func StatefulSetForRack(r scyllav1.RackSpec, c *scyllav1.ScyllaCluster, sidecarI
 								},
 							},
 						},
+						{
+							Name: scyllaAgentAuthTokenVolumeName,
+							VolumeSource: corev1.VolumeSource{
+								Secret: &corev1.SecretVolumeSource{
+									SecretName: naming.AgentAuthTokenSecretName(c.Name),
+								},
+							},
+						},
 					},
 					Tolerations: placement.Tolerations,
 					InitContainers: []corev1.Container{
@@ -454,7 +469,9 @@ func agentContainer(r scyllav1.RackSpec, c *scyllav1.ScyllaCluster) corev1.Conta
 			"-c",
 			naming.ScyllaAgentConfigDefaultFile,
 			"-c",
-			naming.ScyllaAgentConfigDirName + "/scylla-manager-agent.yaml",
+			path.Join(naming.ScyllaAgentConfigDirName, naming.ScyllaAgentConfigFileName),
+			"-c",
+			path.Join(naming.ScyllaAgentConfigDirName, naming.ScyllaAgentAuthTokenFileName),
 		},
 		Ports: []corev1.ContainerPort{
 			{
@@ -468,8 +485,15 @@ func agentContainer(r scyllav1.RackSpec, c *scyllav1.ScyllaCluster) corev1.Conta
 				MountPath: naming.DataDir,
 			},
 			{
-				Name:      "scylla-agent-config-volume",
-				MountPath: naming.ScyllaAgentConfigDirName,
+				Name:      scyllaAgentConfigVolumeName,
+				MountPath: path.Join(naming.ScyllaAgentConfigDirName, naming.ScyllaAgentConfigFileName),
+				SubPath:   naming.ScyllaAgentConfigFileName,
+				ReadOnly:  true,
+			},
+			{
+				Name:      scyllaAgentAuthTokenVolumeName,
+				MountPath: path.Join(naming.ScyllaAgentConfigDirName, naming.ScyllaAgentAuthTokenFileName),
+				SubPath:   naming.ScyllaAgentAuthTokenFileName,
 				ReadOnly:  true,
 			},
 		},
@@ -498,6 +522,25 @@ func MakePodDisruptionBudget(c *scyllav1.ScyllaCluster) *v1beta1.PodDisruptionBu
 	}
 }
 
+func MakeAgentAuthTokenSecret(c *scyllav1.ScyllaCluster, authToken string) (*corev1.Secret, error) {
+	data, err := yaml.Marshal(&helpers.AgentAuthTokenSecret{AuthToken: authToken})
+	if err != nil {
+		return nil, fmt.Errorf("marshal auth token: %w", err)
+	}
+
+	return &corev1.Secret{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:            naming.AgentAuthTokenSecretName(c.Name),
+			Namespace:       c.Namespace,
+			OwnerReferences: []metav1.OwnerReference{util.NewControllerRef(c)},
+		},
+		Type: corev1.SecretTypeOpaque,
+		Data: map[string][]byte{
+			naming.ScyllaAgentAuthTokenFileName: data,
+		},
+	}, nil
+}
+
 func ImageForCluster(c *scyllav1.ScyllaCluster) string {
 	return fmt.Sprintf("%s:%s", c.Spec.Repository, c.Spec.Version)
 }
diff --git a/pkg/controllers/cluster/sync.go b/pkg/controllers/cluster/sync.go
index 1289c39da..8cbf39b2a 100644
--- a/pkg/controllers/cluster/sync.go
+++ b/pkg/controllers/cluster/sync.go
@@ -18,6 +18,7 @@ const (
 	// Messages to display when experiencing an error.
 	MessageHeadlessServiceSyncFailed     = "Failed to sync Headless Service for cluster"
 	MessagePodDisruptionBudgetSyncFailed = "Failed to sync Pod Disruption Budget for cluster"
+	MessageAgentTokenSyncFailed          = "Failed to sync Agent token for cluster"
 	MessageMemberServicesSyncFailed      = "Failed to sync MemberServices for cluster"
 	MessageUpdateStatusFailed            = "Failed to update status for cluster: %+v"
 	MessageCleanupFailed                 = "Failed to clean up cluster resources"
@@ -60,6 +61,12 @@ func (cc *ClusterReconciler) sync(c *scyllav1.ScyllaCluster) error {
 		return errors.Wrap(err, "failed to sync pod disruption budget")
 	}
 
+	// Sync Agent auth token
+	if err := cc.syncAgentAuthToken(ctx, c); err != nil {
+		cc.Recorder.Event(c, corev1.EventTypeWarning, naming.ErrSyncFailed, MessageAgentTokenSyncFailed)
+		return errors.Wrap(err, "failed to sync agent auth token")
+	}
+
 	// Sync Cluster Member Services
 	if err := cc.syncMemberServices(ctx, c); err != nil {
 		cc.Recorder.Event(c, corev1.EventTypeWarning, naming.ErrSyncFailed, MessageMemberServicesSyncFailed)
diff --git a/pkg/controllers/helpers/agent.go b/pkg/controllers/helpers/agent.go
new file mode 100644
index 000000000..6f030c849
--- /dev/null
+++ b/pkg/controllers/helpers/agent.go
@@ -0,0 +1,36 @@
+// Copyright (C) 2021 ScyllaDB
+
+package helpers
+
+import (
+	"context"
+	"fmt"
+
+	"github.com/scylladb/scylla-operator/pkg/naming"
+	"gopkg.in/yaml.v2"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
+)
+
+type AgentAuthTokenSecret struct {
+	AuthToken string `yaml:"auth_token"`
+}
+
+func GetAgentAuthToken(ctx context.Context, client corev1client.CoreV1Interface, clusterName, namespace string) (string, error) {
+	secret, err := client.Secrets(namespace).Get(ctx, naming.AgentAuthTokenSecretName(clusterName), metav1.GetOptions{})
+	if err != nil {
+		return "", fmt.Errorf("get auth token secret: %w", err)
+	}
+
+	return GetAgentAuthTokenFromSecret(secret)
+}
+
+func GetAgentAuthTokenFromSecret(secret *corev1.Secret) (string, error) {
+	data := &AgentAuthTokenSecret{}
+	if err := yaml.Unmarshal(secret.Data[naming.ScyllaAgentAuthTokenFileName], data); err != nil {
+		return "", fmt.Errorf("unmarshal agent auth token secret: %w", err)
+	}
+
+	return data.AuthToken, nil
+}
diff --git a/pkg/controllers/manager/manager_controller.go b/pkg/controllers/manager/manager_controller.go
index 9782d84c3..e15dcfafc 100644
--- a/pkg/controllers/manager/manager_controller.go
+++ b/pkg/controllers/manager/manager_controller.go
@@ -12,14 +12,15 @@ import (
 	"github.com/pkg/errors"
 	"github.com/scylladb/go-log"
 	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
 	"github.com/scylladb/scylla-operator/pkg/mermaidclient"
 	"github.com/scylladb/scylla-operator/pkg/naming"
 	"go.uber.org/multierr"
-	"gopkg.in/yaml.v2"
 	corev1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	"k8s.io/apimachinery/pkg/runtime"
 	"k8s.io/apimachinery/pkg/util/wait"
+	"k8s.io/client-go/kubernetes"
 	ctrl "sigs.k8s.io/controller-runtime"
 	mgr "sigs.k8s.io/controller-runtime/pkg/manager"
 	"sigs.k8s.io/controller-runtime/pkg/reconcile"
@@ -30,6 +31,7 @@ import (
 // Reconciler reconciles a Cluster object
 type Reconciler struct {
 	client.Client
+	KubeClient     kubernetes.Interface
 	UncachedClient client.Client
 	ManagerClient  *mermaidclient.Client
 
@@ -40,6 +42,7 @@ type Reconciler struct {
 }
 
 func New(ctx context.Context, mgr mgr.Manager, ignoredNamespace, namespace string, logger log.Logger) (*Reconciler, error) {
+	kubeClient := kubernetes.NewForConfigOrDie(mgr.GetConfig())
 	c, err := client.New(mgr.GetConfig(), client.Options{
 		Scheme: mgr.GetScheme(),
 	})
@@ -63,6 +66,7 @@ func New(ctx context.Context, mgr mgr.Manager, ignoredNamespace, namespace strin
 
 	return &Reconciler{
 		Client:           mgr.GetClient(),
+		KubeClient:       kubeClient,
 		UncachedClient:   c,
 		ManagerClient:    manager,
 		IgnoredNamespace: ignoredNamespace,
@@ -153,7 +157,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request) (ctrl.Resu
 	}
 	r.Logger.Debug(ctx, "Manager state", "state", managerState)
 
-	authToken, err := r.getAuthToken(ctx, &cluster)
+	authToken, err := helpers.GetAgentAuthToken(ctx, r.KubeClient.CoreV1(), cluster.Name, cluster.Namespace)
 	if err != nil {
 		logger.Error(ctx, "Failed to fetch cluster auth token", "error", err)
 		return reconcile.Result{}, err
@@ -250,31 +254,6 @@ func (r *Reconciler) managerState(ctx context.Context, clusterID string) (*state
 	}, nil
 }
 
-func (r *Reconciler) getAuthToken(ctx context.Context, cluster *scyllav1.ScyllaCluster) (string, error) {
-	const agentConfigKey = "scylla-manager-agent.yaml"
-	secret := &corev1.Secret{}
-	agentSecretName := cluster.Spec.Datacenter.Racks[0].ScyllaAgentConfig
-	secretKey := client.ObjectKey{
-		Name:      agentSecretName,
-		Namespace: cluster.Namespace,
-	}
-	err := r.Client.Get(ctx, secretKey, secret)
-	if err != nil {
-		if apierrors.IsNotFound(err) {
-			return "", nil
-		}
-		return "", errors.Wrapf(err, "get %s secret", agentSecretName)
-	}
-	agentConfig := struct {
-		AuthToken string `yaml:"auth_token"`
-	}{}
-	if err := yaml.Unmarshal(secret.Data[agentConfigKey], &agentConfig); err != nil {
-		return "", errors.Wrap(err, "cannot parse agent config")
-	}
-
-	return agentConfig.AuthToken, nil
-}
-
 func (r *Reconciler) SetupWithManager(mgr ctrl.Manager) error {
 	return ctrl.NewControllerManagedBy(mgr).
 		For(&scyllav1.ScyllaCluster{}).
diff --git a/pkg/controllers/sidecar/sidecar_controller.go b/pkg/controllers/sidecar/sidecar_controller.go
index 5fb953d8a..c856d0d04 100644
--- a/pkg/controllers/sidecar/sidecar_controller.go
+++ b/pkg/controllers/sidecar/sidecar_controller.go
@@ -18,11 +18,13 @@ package sidecar
 import (
 	"context"
 	"fmt"
+	"path"
 	"time"
 
 	"github.com/pkg/errors"
 	"github.com/scylladb/go-log"
 	"github.com/scylladb/scylla-operator/pkg/cmd/operator/options"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
 	"github.com/scylladb/scylla-operator/pkg/controllers/sidecar/config"
 	"github.com/scylladb/scylla-operator/pkg/controllers/sidecar/identity"
 	"github.com/scylladb/scylla-operator/pkg/naming"
@@ -106,10 +108,18 @@ func New(ctx context.Context, mgr manager.Manager, logger log.Logger) (*MemberRe
 		return nil, errors.Wrap(err, "get scylla address")
 	}
 
-	cfg := scyllaclient.DefaultConfig(host.String())
-	if err := cfgutil.ParseYAML(&cfg, naming.ScyllaClientConfigDirName+"/"+naming.ScyllaClientConfigFileName); err != nil {
+	authToken, err := helpers.GetAgentAuthToken(ctx, kubeClient.CoreV1(), member.Cluster, member.Namespace)
+	if err != nil {
+		return nil, errors.Wrap(err, "get auth token")
+	}
+
+	cfg := scyllaclient.DefaultConfig(authToken, host.String())
+	if err := cfgutil.ParseYAML(&cfg, path.Join(naming.ScyllaClientConfigDirName, naming.ScyllaClientConfigFileName)); err != nil {
 		return nil, errors.Wrap(err, "parse scylla agent config")
 	}
+	// Auth token from secret is preferred.
+	cfg.AuthToken = authToken
+
 	scyllaClient, err := scyllaclient.NewClient(cfg, logger.Named("scylla_client"))
 	if err != nil {
 		return nil, errors.Wrap(err, "create scylla client")
diff --git a/pkg/naming/constants.go b/pkg/naming/constants.go
index aef7b22d7..b1591dc10 100644
--- a/pkg/naming/constants.go
+++ b/pkg/naming/constants.go
@@ -74,6 +74,8 @@ const (
 
 	ScyllaConfigDirName          = "/mnt/scylla-config"
 	ScyllaAgentConfigDirName     = "/mnt/scylla-agent-config"
+	ScyllaAgentConfigFileName    = "scylla-manager-agent.yaml"
+	ScyllaAgentAuthTokenFileName = "auth-token.yaml"
 	ScyllaAgentConfigDefaultFile = "/etc/scylla-manager-agent/scylla-manager-agent.yaml"
 	ScyllaClientConfigDirName    = "/mnt/scylla-client-config"
 	ScyllaClientConfigFileName   = "scylla-client.yaml"
diff --git a/pkg/naming/names.go b/pkg/naming/names.go
index 69f60bb64..0c83d52a1 100644
--- a/pkg/naming/names.go
+++ b/pkg/naming/names.go
@@ -35,6 +35,10 @@ func ServiceNameFromPod(pod *corev1.Pod) string {
 	return pod.Name
 }
 
+func AgentAuthTokenSecretName(clusterName string) string {
+	return fmt.Sprintf("%s-auth-token", clusterName)
+}
+
 func MemberServiceName(r scyllav1.RackSpec, c *scyllav1.ScyllaCluster, idx int) string {
 	return fmt.Sprintf("%s-%d", StatefulSetNameForRack(r, c), idx)
 }
diff --git a/pkg/scyllaclient/config.go b/pkg/scyllaclient/config.go
index 8e128c927..bfcc7da41 100644
--- a/pkg/scyllaclient/config.go
+++ b/pkg/scyllaclient/config.go
@@ -47,12 +47,13 @@ type BackoffConfig struct {
 }
 
 // DefaultConfig returns a Config initialized with default values.
-func DefaultConfig(hosts ...string) Config {
+func DefaultConfig(authToken string, hosts ...string) Config {
 	return Config{
-		Hosts:   hosts,
-		Port:    "10001",
-		Scheme:  "https",
-		Timeout: 15 * time.Second,
+		AuthToken: authToken,
+		Hosts:     hosts,
+		Port:      "10001",
+		Scheme:    "https",
+		Timeout:   15 * time.Second,
 		Backoff: BackoffConfig{
 			WaitMin:    1 * time.Second,
 			WaitMax:    30 * time.Second,
diff --git a/pkg/util/resourceapply/secret.go b/pkg/util/resourceapply/secret.go
new file mode 100644
index 000000000..b6729d97e
--- /dev/null
+++ b/pkg/util/resourceapply/secret.go
@@ -0,0 +1,54 @@
+// Copyright (C) 2021 ScyllaDB
+
+package resourceapply
+
+import (
+	"context"
+
+	"github.com/pkg/errors"
+	"github.com/scylladb/scylla-operator/pkg/naming"
+	corev1 "k8s.io/api/core/v1"
+	apierrors "k8s.io/apimachinery/pkg/api/errors"
+	"k8s.io/client-go/tools/record"
+	"sigs.k8s.io/controller-runtime/pkg/client"
+)
+
+// ApplySecret creates or updates Secret. If desired object is equal (based on calculated hash)
+// to existing one update is not called.
+func ApplySecret(ctx context.Context, recorder record.EventRecorder, client client.Client, secret *corev1.Secret) error {
+	required := secret.DeepCopy()
+	if err := setHash(required); err != nil {
+		return errors.Wrap(err, "set hash on secret")
+	}
+
+	existing := &corev1.Secret{}
+	err := client.Get(ctx, naming.NamespacedName(required.Name, required.Namespace), existing)
+	if err != nil {
+		if !apierrors.IsNotFound(err) {
+			return errors.Wrap(err, "get secret")
+		}
+
+		err := client.Create(ctx, required)
+		if err != nil {
+			ReportCreateEvent(recorder, required, err)
+			return errors.Wrap(err, "create secret")
+		}
+
+		ReportCreateEvent(recorder, required, nil)
+		return nil
+	}
+
+	// If they are the same do nothing.
+	if existing.Annotations[naming.ManagedHash] == required.Annotations[naming.ManagedHash] {
+		return nil
+	}
+
+	required.ResourceVersion = existing.ResourceVersion
+	if err := client.Update(ctx, required); err != nil {
+		ReportUpdateEvent(recorder, secret, err)
+		return errors.Wrap(err, "update secret")
+	}
+
+	ReportUpdateEvent(recorder, secret, nil)
+	return nil
+}


Test patch:
diff --git a/pkg/util/resourceapply/secret_test.go b/pkg/util/resourceapply/secret_test.go
new file mode 100644
index 000000000..47b2dfbea
--- /dev/null
+++ b/pkg/util/resourceapply/secret_test.go
@@ -0,0 +1,109 @@
+// Copyright (C) 2021 ScyllaDB
+
+package resourceapply_test
+
+import (
+	"context"
+	"testing"
+
+	"github.com/scylladb/scylla-operator/pkg/naming"
+	"github.com/scylladb/scylla-operator/pkg/util/resourceapply"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/client-go/tools/record"
+	"sigs.k8s.io/controller-runtime/pkg/client/fake"
+)
+
+func TestApplySecret(t *testing.T) {
+	ctx := context.Background()
+
+	secret := &corev1.Secret{
+		ObjectMeta: metav1.ObjectMeta{
+			Annotations: make(map[string]string),
+			Name:        "secret-name",
+			Namespace:   "secret-namespace",
+		},
+		Type: corev1.SecretTypeOpaque,
+		Data: map[string][]byte{
+			"key": []byte("value"),
+		},
+	}
+
+	secretWithHash := secret.DeepCopy()
+	secretWithHash.Annotations[naming.ManagedHash] = resourceapply.MustHashObjects(secretWithHash)
+
+	differentSecret := secret.DeepCopy()
+	differentSecret.Data = map[string][]byte{
+		"yek": []byte("eulav"),
+	}
+
+	tests := []struct {
+		Name         string
+		Objects      []runtime.Object
+		Secret       *corev1.Secret
+		ExpectedHash string
+		ExpectEvent  bool
+	}{
+		{
+			Name:         "object is created when it doesn't exists",
+			Objects:      []runtime.Object{},
+			Secret:       secret.DeepCopy(),
+			ExpectedHash: resourceapply.MustHashObjects(secret.DeepCopy()),
+			ExpectEvent:  true,
+		},
+		{
+			Name:         "object isn't updated when it has the same hash",
+			Objects:      []runtime.Object{secretWithHash.DeepCopy()},
+			Secret:       secret.DeepCopy(),
+			ExpectedHash: secretWithHash.Annotations[naming.ManagedHash],
+			ExpectEvent:  false,
+		},
+		{
+			Name:         "object is updated when it is different than existing one",
+			Objects:      []runtime.Object{secretWithHash.DeepCopy()},
+			Secret:       differentSecret.DeepCopy(),
+			ExpectedHash: resourceapply.MustHashObjects(differentSecret.DeepCopy()),
+			ExpectEvent:  true,
+		},
+	}
+
+	for i := range tests {
+		test := tests[i]
+		t.Run(test.Name, func(t *testing.T) {
+			t.Parallel()
+
+			client := fake.NewClientBuilder().
+				WithScheme(ClientGoScheme()).
+				WithRuntimeObjects(test.Objects...).
+				Build()
+			recorder := record.NewFakeRecorder(10)
+
+			if err := resourceapply.ApplySecret(ctx, recorder, client, test.Secret); err != nil {
+				t.Errorf("failed to apply secret, got err %v", err)
+			}
+
+			secretKey := naming.NamespacedName(test.Secret.Name, test.Secret.Namespace)
+			result := &corev1.Secret{}
+			if err := client.Get(ctx, secretKey, result); err != nil {
+				t.Errorf("failed to get secret, got err %v", err)
+			}
+
+			if result.Annotations[naming.ManagedHash] != test.ExpectedHash {
+				t.Errorf("object hash is different then expected %s", result.Annotations[naming.ManagedHash])
+			}
+
+			if test.ExpectEvent {
+				select {
+				case _ = <-recorder.Events:
+				default:
+					t.Errorf("expected event but it wasn't emitted")
+				}
+			} else {
+				if len(recorder.Events) != 0 {
+					t.Errorf("expected no event but it was emitted, %s", <-recorder.Events)
+				}
+			}
+		})
+	}
+}
diff --git a/test/e2e/set/scyllacluster/helpers.go b/test/e2e/set/scyllacluster/helpers.go
index c403c057f..24af0d098 100644
--- a/test/e2e/set/scyllacluster/helpers.go
+++ b/test/e2e/set/scyllacluster/helpers.go
@@ -10,9 +10,11 @@ import (
 	"github.com/scylladb/go-log"
 	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
 	scyllav1client "github.com/scylladb/scylla-operator/pkg/client/scylla/clientset/versioned/typed/scylla/v1"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
 	"github.com/scylladb/scylla-operator/pkg/mermaidclient"
 	"github.com/scylladb/scylla-operator/pkg/naming"
 	"github.com/scylladb/scylla-operator/pkg/scyllaclient"
+	"github.com/scylladb/scylla-operator/test/e2e/scheme"
 	"go.uber.org/zap"
 	"go.uber.org/zap/zapcore"
 	appsv1 "k8s.io/api/apps/v1"
@@ -21,6 +23,8 @@ import (
 	"k8s.io/apimachinery/pkg/fields"
 	"k8s.io/apimachinery/pkg/labels"
 	"k8s.io/apimachinery/pkg/runtime"
+	"k8s.io/apimachinery/pkg/types"
+	"k8s.io/apimachinery/pkg/util/strategicpatch"
 	"k8s.io/apimachinery/pkg/watch"
 	appv1client "k8s.io/client-go/kubernetes/typed/apps/v1"
 	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
@@ -106,6 +110,35 @@ func waitForScyllaClusterState(ctx context.Context, client scyllav1client.Scylla
 	return event.Object.(*scyllav1.ScyllaCluster), nil
 }
 
+func waitForStatefulSetRollout(ctx context.Context, client appv1client.AppsV1Interface, namespace, name string) (*appsv1.StatefulSet, error) {
+	fieldSelector := fields.OneTermEqualSelector("metadata.name", name).String()
+	lw := &cache.ListWatch{
+		ListFunc: func(options metav1.ListOptions) (runtime.Object, error) {
+			options.FieldSelector = fieldSelector
+			return client.StatefulSets(namespace).List(ctx, options)
+		},
+		WatchFunc: func(options metav1.ListOptions) (i watch.Interface, e error) {
+			options.FieldSelector = fieldSelector
+			return client.StatefulSets(namespace).Watch(ctx, options)
+		},
+	}
+	event, err := watchtools.UntilWithSync(ctx, lw, &appsv1.StatefulSet{}, nil, func(event watch.Event) (bool, error) {
+		switch event.Type {
+		case watch.Added, watch.Modified:
+			sts := event.Object.(*appsv1.StatefulSet)
+			return sts.Status.ObservedGeneration >= sts.Generation &&
+				sts.Status.CurrentRevision == sts.Status.UpdateRevision &&
+				sts.Status.ReadyReplicas == *sts.Spec.Replicas, nil
+		default:
+			return true, fmt.Errorf("unexpected event: %#v", event)
+		}
+	})
+	if err != nil {
+		return nil, err
+	}
+	return event.Object.(*appsv1.StatefulSet), nil
+}
+
 type waitForStateOptions struct {
 	tolerateDelete bool
 }
@@ -201,14 +234,18 @@ func getScyllaClient(ctx context.Context, client corev1client.CoreV1Interface, s
 		return nil, nil, fmt.Errorf("no services found")
 	}
 
+	authToken, err := helpers.GetAgentAuthToken(ctx, client, sc.Name, sc.Namespace)
+	if err != nil {
+		return nil, nil, fmt.Errorf("get auth token: %w", err)
+	}
+
 	// TODO: unify logging
 	logger, _ := log.NewProduction(log.Config{
 		Level: zap.NewAtomicLevelAt(zapcore.InfoLevel),
 	})
-	scyllaClient, err := scyllaclient.NewClient(
-		scyllaclient.DefaultConfig(hosts...),
-		logger.Named("scylla_client"),
-	)
+	cfg := scyllaclient.DefaultConfig(authToken, hosts...)
+
+	scyllaClient, err := scyllaclient.NewClient(cfg, logger.Named("scylla_client"))
 	if err != nil {
 		return nil, nil, err
 	}
@@ -252,3 +289,36 @@ func getNodeName(sc *scyllav1.ScyllaCluster, idx int) string {
 		idx,
 	)
 }
+
+func restartRack(ctx context.Context, client appv1client.AppsV1Interface, r scyllav1.RackSpec, sc *scyllav1.ScyllaCluster) error {
+	sts, err := client.StatefulSets(sc.Namespace).Get(ctx, naming.StatefulSetNameForRack(r, sc), metav1.GetOptions{})
+	if err != nil {
+		return err
+	}
+
+	before, err := runtime.Encode(scheme.DefaultJSONEncoder(), sts)
+	if err != nil {
+		return err
+	}
+
+	if sts.Spec.Template.ObjectMeta.Annotations == nil {
+		sts.Spec.Template.ObjectMeta.Annotations = make(map[string]string)
+	}
+	sts.Spec.Template.ObjectMeta.Annotations["e2e.scylladb.com/restartedAt"] = time.Now().Format(time.RFC3339)
+
+	after, err := runtime.Encode(scheme.DefaultJSONEncoder(), sts)
+	if err != nil {
+		return err
+	}
+	patch, err := strategicpatch.CreateTwoWayMergePatch(before, after, sts)
+	if err != nil {
+		return err
+	}
+
+	_, err = client.StatefulSets(sc.Namespace).Patch(ctx, sts.Name, types.StrategicMergePatchType, patch, metav1.PatchOptions{})
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
diff --git a/test/e2e/set/scyllacluster/scyllacluster_auth.go b/test/e2e/set/scyllacluster/scyllacluster_auth.go
new file mode 100644
index 000000000..97c295256
--- /dev/null
+++ b/test/e2e/set/scyllacluster/scyllacluster_auth.go
@@ -0,0 +1,155 @@
+// Copyright (C) 2021 ScyllaDB
+
+package scyllacluster
+
+import (
+	"context"
+	"fmt"
+	"net/http"
+
+	g "github.com/onsi/ginkgo"
+	o "github.com/onsi/gomega"
+	"github.com/scylladb/go-log"
+	"github.com/scylladb/scylla-operator/pkg/controllers/helpers"
+	"github.com/scylladb/scylla-operator/pkg/naming"
+	"github.com/scylladb/scylla-operator/pkg/scyllaclient"
+	scyllaclusterfixture "github.com/scylladb/scylla-operator/test/e2e/fixture/scyllacluster"
+	"github.com/scylladb/scylla-operator/test/e2e/framework"
+	"gopkg.in/yaml.v2"
+	corev1 "k8s.io/api/core/v1"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	"k8s.io/apimachinery/pkg/types"
+)
+
+var _ = g.Describe("ScyllaCluster authentication", func() {
+	defer g.GinkgoRecover()
+
+	f := framework.NewFramework("scyllacluster")
+
+	g.It("agent requires authentication", func() {
+		ctx, cancel := context.WithTimeout(context.Background(), testTimout)
+		defer cancel()
+
+		sc := scyllaclusterfixture.BasicScyllaCluster.ReadOrFail()
+		sc.Spec.Datacenter.Racks[0].Members = 1
+
+		framework.By("Creating a ScyllaCluster")
+		err := framework.SetupScyllaClusterSA(ctx, f.KubeClient().CoreV1(), f.KubeClient().RbacV1(), f.Namespace(), sc.Name)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		sc, err = f.ScyllaClient().ScyllaV1().ScyllaClusters(f.Namespace()).Create(ctx, sc, metav1.CreateOptions{})
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Waiting for the ScyllaCluster to deploy")
+		waitCtx1, waitCtx1Cancel := contextForRollout(ctx, sc)
+		defer waitCtx1Cancel()
+		sc, err = waitForScyllaClusterState(waitCtx1, f.ScyllaClient().ScyllaV1(), sc.Namespace, sc.Name, scyllaClusterRolledOut)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		verifyScyllaCluster(ctx, f.KubeClient(), sc)
+
+		framework.By("Rejecting an unauthorized request")
+
+		_, hosts, err := getScyllaClient(ctx, f.KubeClient().CoreV1(), sc)
+
+		emptyAuthToken := ""
+		_, err = getScyllaClientStatus(ctx, hosts, emptyAuthToken)
+		o.Expect(err).To(o.HaveOccurred())
+		o.Expect(scyllaclient.StatusCodeOf(err)).To(o.Equal(http.StatusUnauthorized))
+
+		framework.By("Accepting requests authorized using token from provisioned secret")
+
+		token, err := helpers.GetAgentAuthToken(ctx, f.KubeClient().CoreV1(), sc.Name, sc.Namespace)
+		o.Expect(err).ToNot(o.HaveOccurred())
+
+		_, err = getScyllaClientStatus(ctx, hosts, token)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Specifying auth token in agent config")
+
+		agentConfig := struct {
+			AuthToken string `yaml:"auth_token"`
+		}{
+			AuthToken: "42",
+		}
+		agentConfigData, err := yaml.Marshal(agentConfig)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		agentConfigSecret := &corev1.Secret{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "agent-config",
+			},
+			Type: corev1.SecretTypeOpaque,
+			Data: map[string][]byte{
+				naming.ScyllaAgentConfigFileName: agentConfigData,
+			},
+		}
+
+		_, err = f.KubeClient().CoreV1().Secrets(f.Namespace()).Create(ctx, agentConfigSecret, metav1.CreateOptions{})
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		_, err = f.ScyllaClient().ScyllaV1().ScyllaClusters(f.Namespace()).Patch(
+			ctx,
+			sc.Name,
+			types.JSONPatchType,
+			[]byte(fmt.Sprintf(`[{"op":"replace","path":"/spec/datacenter/racks/0/scyllaAgentConfig","value":"%s"}]`, agentConfigSecret.Name)),
+			metav1.PatchOptions{},
+		)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		// TODO: restart should be triggered by the Operator
+		framework.By("Initiating a rolling restart")
+
+		err = restartRack(ctx, f.KubeClient().AppsV1(), sc.Spec.Datacenter.Racks[0], sc)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Waiting for the ScyllaCluster to pick up token change")
+
+		stss, err := getStatefulSetsForScyllaCluster(ctx, f.KubeClient().AppsV1(), sc)
+		o.Expect(err).NotTo(o.HaveOccurred())
+		rackSts := stss[sc.Spec.Datacenter.Racks[0].Name]
+
+		waitCtx2, waitCtx2Cancel := contextForRollout(ctx, sc)
+		defer waitCtx2Cancel()
+		_, err = waitForStatefulSetRollout(waitCtx2, f.KubeClient().AppsV1(), rackSts.Namespace, rackSts.Name)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Accepting requests authorized using token from user agent config")
+		_, err = getScyllaClientStatus(ctx, hosts, agentConfig.AuthToken)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Changing auth token in agent config")
+
+		agentConfig.AuthToken = "666"
+		agentConfigData, err = yaml.Marshal(agentConfig)
+		o.Expect(err).NotTo(o.HaveOccurred())
+		agentConfigSecret.Data[naming.ScyllaAgentConfigFileName] = agentConfigData
+		_, err = f.KubeClient().CoreV1().Secrets(f.Namespace()).Update(ctx, agentConfigSecret, metav1.UpdateOptions{})
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Initiating a rolling restart")
+
+		err = restartRack(ctx, f.KubeClient().AppsV1(), sc.Spec.Datacenter.Racks[0], sc)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Waiting for the ScyllaCluster to pick up token change")
+
+		waitCtx3, waitCtx3Cancel := contextForRollout(ctx, sc)
+		defer waitCtx3Cancel()
+		_, err = waitForStatefulSetRollout(waitCtx3, f.KubeClient().AppsV1(), rackSts.Namespace, rackSts.Name)
+		o.Expect(err).NotTo(o.HaveOccurred())
+
+		framework.By("Accepting requests authorized using token from user agent config")
+		_, err = getScyllaClientStatus(ctx, hosts, agentConfig.AuthToken)
+		o.Expect(err).NotTo(o.HaveOccurred())
+	})
+})
+
+func getScyllaClientStatus(ctx context.Context, hosts []string, authToken string) (scyllaclient.NodeStatusInfoSlice, error) {
+	cfg := scyllaclient.DefaultConfig(authToken, hosts...)
+
+	client, err := scyllaclient.NewClient(cfg, log.NopLogger)
+	o.Expect(err).NotTo(o.HaveOccurred())
+
+	return client.Status(ctx, hosts[0])
+}
diff --git a/test/integration/upgrade_version_test.go b/test/integration/upgrade_version_test.go
index 7a5e1923e..8f57284ff 100644
--- a/test/integration/upgrade_version_test.go
+++ b/test/integration/upgrade_version_test.go
@@ -20,6 +20,7 @@ import (
 	"github.com/scylladb/scylla-operator/pkg/util/httpx"
 	corev1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
+	corev1client "k8s.io/client-go/kubernetes/typed/core/v1"
 	"sigs.k8s.io/controller-runtime/pkg/client"
 
 	scyllav1 "github.com/scylladb/scylla-operator/pkg/api/scylla/v1"
@@ -109,7 +110,7 @@ var _ = Describe("Cluster controller", func() {
 			scylla  *scyllav1.ScyllaCluster
 			sstStub *integration.StatefulSetOperatorStub
 
-			originalActionsScyllaClientForClusterFunc func(ctx context.Context, cc client.Client, hosts []string, logger log.Logger) (*scyllaclient.Client, error)
+			originalActionsScyllaClientForClusterFunc func(ctx context.Context, client corev1client.CoreV1Interface, cluster *scyllav1.ScyllaCluster, hosts []string, logger log.Logger) (*scyllaclient.Client, error)
 		)
 
 		BeforeEach(func() {
@@ -179,8 +180,8 @@ var _ = Describe("Cluster controller", func() {
 				return http.DefaultClient.Do(req)
 			}))
 
-			actions.ScyllaClientForClusterFunc = func(ctx context.Context, cc client.Client, hosts []string, logger log.Logger) (*scyllaclient.Client, error) {
-				cfg := scyllaclient.DefaultConfig(scyllaAddr)
+			actions.ScyllaClientForClusterFunc = func(ctx context.Context, client corev1client.CoreV1Interface, cluster *scyllav1.ScyllaCluster, hosts []string, logger log.Logger) (*scyllaclient.Client, error) {
+				cfg := scyllaclient.DefaultConfig("", scyllaAddr)
 				cfg.Transport = hrt
 				return scyllaclient.NewClient(cfg, logger)
 			}