Repository: canonical/microk8s

Issue:
microk8s.status shows "...var/kubernetes/backend/cluster.yaml: permission denied" error
Hello,

I installed microk8s and saw a temporary error in `microk8s.status`.

These are the logs:

```shell
$ sudo snap install microk8s --classic
[sudo] password for davigar15: 
microk8s (1.23/stable) v1.23.3 from Canonical✓ installed
$ microk8s.status 
microk8s is running
Error: open servers store: open /var/snap/microk8s/2948/var/kubernetes/backend/cluster.yaml: permission denied
Usage:
  dqlite -s <servers> <database> [command] [flags]

Flags:
  -c, --cert string       public TLS cert
  -f, --format string     output format (tabular, json) (default "tabular")
  -h, --help              help for dqlite
  -k, --key string        private TLS key
  -s, --servers strings   comma-separated list of db servers, or file://<store>

mhigh-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    ha-cluster           # Configure high availability on the current node
  disabled:
    ambassador           # Ambassador API Gateway and Ingress
    cilium               # SDN, fast with full network policy
    dashboard            # The Kubernetes dashboard
    dashboard-ingress    # Ingress definition for Kubernetes dashboard
    dns                  # CoreDNS
    fluentd              # Elasticsearch-Fluentd-Kibana logging and monitoring
    gpu                  # Automatic enablement of Nvidia CUDA
    helm                 # Helm 2 - the package manager for Kubernetes
    helm3                # Helm 3 - Kubernetes package manager
    host-access          # Allow Pods connecting to Host services smoothly
    inaccel              # Simplifying FPGA management in Kubernetes
    ingress              # Ingress controller for external access
    istio                # Core Istio service mesh services
    jaeger               # Kubernetes Jaeger operator with its simple config
    kata                 # Kata Containers is a secure runtime with lightweight VMS
    keda                 # Kubernetes-based Event Driven Autoscaling
    knative              # The Knative framework on Kubernetes.
    kubeflow             # Kubeflow for easy ML deployments
    linkerd              # Linkerd is a service mesh for Kubernetes and other frameworks
    metallb              # Loadbalancer for your Kubernetes cluster
    metrics-server       # K8s Metrics Server for API access to service metrics
    multus               # Multus CNI enables attaching multiple network interfaces to pods
    openebs              # OpenEBS is the open-source storage solution for Kubernetes
    openfaas             # OpenFaaS serverless framework
    portainer            # Portainer UI for your Kubernetes cluster
    prometheus           # Prometheus operator for monitoring and logging
    rbac                 # Role-Based Access Control for authorisation
    registry             # Private image registry exposed on localhost:32000
    storage              # Storage class; allocates storage from host directory
    traefik              # traefik Ingress controller for external access
$ microk8s.enable storage dns
Enabling default storage class
deployment.apps/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
serviceaccount/microk8s-hostpath created
clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created
clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created
Storage will be available soon
Enabling DNS
Applying manifest
serviceaccount/coredns created
configmap/coredns created
deployment.apps/coredns created
service/kube-dns created
clusterrole.rbac.authorization.k8s.io/coredns created
clusterrolebinding.rbac.authorization.k8s.io/coredns created
Restarting kubelet
DNS is enabled
$ microk8s.status 
microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    storage              # Storage class; allocates storage from host directory
  disabled:
    ambassador           # Ambassador API Gateway and Ingress
    cilium               # SDN, fast with full network policy
    dashboard            # The Kubernetes dashboard
    dashboard-ingress    # Ingress definition for Kubernetes dashboard
    fluentd              # Elasticsearch-Fluentd-Kibana logging and monitoring
    gpu                  # Automatic enablement of Nvidia CUDA
    helm                 # Helm 2 - the package manager for Kubernetes
    helm3                # Helm 3 - Kubernetes package manager
    host-access          # Allow Pods connecting to Host services smoothly
    inaccel              # Simplifying FPGA management in Kubernetes
    ingress              # Ingress controller for external access
    istio                # Core Istio service mesh services
    jaeger               # Kubernetes Jaeger operator with its simple config
    kata                 # Kata Containers is a secure runtime with lightweight VMS
    keda                 # Kubernetes-based Event Driven Autoscaling
    knative              # The Knative framework on Kubernetes.
    kubeflow             # Kubeflow for easy ML deployments
    linkerd              # Linkerd is a service mesh for Kubernetes and other frameworks
    metallb              # Loadbalancer for your Kubernetes cluster
    metrics-server       # K8s Metrics Server for API access to service metrics
    multus               # Multus CNI enables attaching multiple network interfaces to pods
    openebs              # OpenEBS is the open-source storage solution for Kubernetes
    openfaas             # OpenFaaS serverless framework
    portainer            # Portainer UI for your Kubernetes cluster
    prometheus           # Prometheus operator for monitoring and logging
    rbac                 # Role-Based Access Control for authorisation
    registry             # Private image registry exposed on localhost:32000
    traefik              # traefik Ingress controller for external access
```


Discussion:
Thanks for creating the issue @davigar15 . This looks to be a transient error due to the permissions of the dqlite cluster.yaml file.

There is a delay between updates to the file and updating the permissions, so you may end up with these two situations:

```
ubuntu@magnus ~ $ ls -la /var/snap/microk8s/x1/var/kubernetes/backend/cluster.yaml
-rw------- 1 root root 63 Feb 11 09:08 /var/snap/microk8s/x1/var/kubernetes/backend/cluster.yaml

ubuntu@magnus ~ $ ls -la /var/snap/microk8s/x1/var/kubernetes/backend/cluster.yaml
-rw-rw---- 1 root microk8s 63 Feb 11 09:08 /var/snap/microk8s/x1/var/kubernetes/backend/cluster.yaml
```

The error appears while the file is in `0600` and owned by `root:root`.

We will have to fix this, but it is useful to know that this is in fact harmless, and can be mitigated by retrying the `microk8s status` command.

Resolving patch:
diff --git a/scripts/wrappers/common/utils.py b/scripts/wrappers/common/utils.py
--- a/scripts/wrappers/common/utils.py
+++ b/scripts/wrappers/common/utils.py
@@ -94,6 +94,7 @@ def get_dqlite_info():
                         snappath=snap_path, dbdir=cluster_dir
                     ).split(),
                     timeout=4,
+                    stderr=subprocess.DEVNULL,
                 )
                 if data["Address"] in out.decode():
                     break


Test patch: