Space boxing user accounts with Kiosk
Tags: kubernetes, multitenancy, tips
Published: 2022-07-03 02:19:17

##### What
[Kiosk](https://github.com/loft-sh/kiosk) is a CRD based approach to isolate namespaces with:
- wrapper over namespace called "spaces" which are isolated and resoure quota applied
- wrapper over users (and service-accounts) called "accounts" who operate within these spaces 
- configuration templates like AccountQuota and Template to generalize and simplify the management of above

##### Architecture

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s66xl5j5kuu0iurgk7ea.png)
> Credit: https://github.com/loft-sh/kiosk

##### Who can do it
All of kiosk's setup are to be done by ClusterAdmin (or users with sufficient RBAC permissions)
0. Check you are admin
```bash
❯ kubectl auth can-i "*" "*" --all-namespaces
kubectl auth can-i "*" namespace
kubectl auth can-i "*" clusterrole
kubectl auth can-i "*" crd
yes
yes
yes
yes
```

##### Setting up kiosk and impersonating users
1. Install kiosk
```bash 
# Install kiosk with helm v3
❯ 
kubectl create namespace kiosk
helm install kiosk --repo https://charts.devspace.sh/ kiosk --namespace kiosk --atomic
namespace/kiosk created
NAME: kiosk
...
Learn more about using kiosk here: https://github.com/loft-sh/kiosk#getting-started

#verify
❯ kubectl get pod -n kiosk
NAME                     READY   STATUS    RESTARTS   AGE
kiosk-66dbfcf6db-5rfx2   1/1     Running   0          2m18s
```

3. Create some accounts
```bash
❯ cat account.yaml
apiVersion: tenancy.kiosk.sh/v1alpha1
kind: Account
metadata:
  name: ambers-account
spec:
  subjects:
  - kind: User
    name: amber
    apiGroup: rbac.authorization.k8s.io
---
apiVersion: tenancy.kiosk.sh/v1alpha1
kind: Account
metadata:
  name: blues-account
spec:
  subjects:
  - kind: User
    name: blue
    apiGroup: rbac.authorization.k8s.io
---
❯ kubectl apply -f account.yaml
account.tenancy.kiosk.sh/ambers-account created
account.tenancy.kiosk.sh/blues-account created

# Check RBAC
❯ kubectl get account.tenancy.kiosk.sh
NAME             SPACES   CREATED AT
ambers-account   0        2022-07-03T01:16:41Z
blues-account    0        2022-07-03T01:16:41Z
❯ kubectl get accounts --as=amber
NAME             SPACES   CREATED AT
ambers-account   0        2022-07-03T01:16:41Z
```

---
**Note**
In the real world, users come through external systems and are usually authenticated through systems like Dex (alternatively, if you are in a public cloud, you may be able to use provider-specific solutions such as AWS IAM for EKS or GCP IAM for GKE).
For service-accounts instead of actual users check [this](https://github.com/loft-sh/kiosk#using-serviceaccounts-for-authentication)

---
##### Working with Spaces
Spaces are wrappers over namespaces. Users are allowed to use and operate on spaces that they are alloted to while others
are denied and invisible to them.
```bash
❯ cat spaces.yaml
apiVersion: tenancy.kiosk.sh/v1alpha1
kind: Space
metadata:
  name: ambers-space
spec:
  # spec.account can be omitted if the current user only belongs to a single account
  account: ambers-account
---
apiVersion: tenancy.kiosk.sh/v1alpha1
kind: Space
metadata:
  name: blues-space
spec:
  # spec.account can be omitted if the current user only belongs to a single account
  account: blues-account
---
❯ k create -f spaces.yaml
space.tenancy.kiosk.sh/ambers-space created
space.tenancy.kiosk.sh/blues-space created
# Verify 
❯ k get spaces --as=amber
NAME           OWNER            CREATED AT
ambers-space   ambers-account   2022-07-03T01:22:44Z
```

Create resources
```bash
❯ k create deployment test-dep-001 --image=nginx --as=amber --namespace=ambers-space
deployment.apps/test-dep-001 created
❯ k create deployment test-dep-002 --image=nginx --as=amber --namespace=blues-space
error: failed to create deployment: deployments.apps is forbidden: User "amber" cannot create resource "deployments" in API group "apps" in the namespace "blues-space"

# Verify
❯ k get deployments.apps -n ambers-space
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
test-dep-001   0/1     1            0           28s
❯ k get deployments.apps -n blues-space
No resources found in blues-space namespace.
```

##### Delete resources
```bash
❯ k delete space blues-space --as=blue
Error from server (Forbidden): spaces.tenancy.kiosk.sh "blues-space" is forbidden: User "blue" cannot delete resource "spaces" in API group "tenancy.kiosk.sh" at the cluster scope
❯ k delete space blues-space
space.tenancy.kiosk.sh "blues-space" deleted
```
- [Allow account-users to create and delete spaces](https://github.com/loft-sh/kiosk#35-create-deletable-spaces) using  


##### Templates
Templates in kiosk are used to initialize namespaces and apply common resources across namespaces (e.g. secrets). 
When creating a Space, kiosk will use these Templates to populate the newly created Namespace for this Space. 
Templates:
 - can contain one or more kubernetes manifests or a helm invocation
 - administered and managed by TemplateInstance (in the respective namespace)
 - can be parametrized for customizability

Create a manifest
- [sample](https://raw.githubusercontent.com/kiosk-sh/kiosk/master/examples/template-manifests.yaml)

Associate with an account
- [sample](https://raw.githubusercontent.com/kiosk-sh/kiosk/master/examples/account-default-template.yaml)

Instantiate the template
- [sample](https://raw.githubusercontent.com/kiosk-sh/kiosk/master/examples/space-template-mandatory.yaml --as=john)

**[Way more information on templates]**(https://github.com/loft-sh/kiosk#53-using-templates)

##### Miscellaneous
1. Account defaults
```bash
❯ cat account-default-space-metadata.yaml
apiVersion: tenancy.kiosk.sh/v1alpha1
kind: Account
metadata:
  name: alpha-space-default-metadata
spec:
  space:
    clusterRole: kiosk-space-admin
    spaceTemplate:
      metadata:
        labels:
          some-label: "label-value"
          other-label: "other-value"
        annotations:
          "foo": "bar"
          "department": "alpha"
  subjects:
  - kind: User
    name: adam
    apiGroup: rbac.authorization.k8s.io
  - kind: User
    name: brian
    apiGroup: rbac.authorization.k8s.io
❯ k create -f account-default-space-metadata.yaml
account.tenancy.kiosk.sh/alpha-space-default-metadata created
❯ k describe account alpha-space-default-metadata 
...
Name: alpha-space-default-metadata
Namespace: null
Spec:
  Space:
    Cluster Role: kiosk-space-admin
    Space Template:
      Metadata:
        Annotations:
          Department: alpha
          Foo: bar
        Creation Timestamp: <nil>
        Labels:
          Other - Label: other-value
          Some - Label: label-value
  Subjects:
    API Group: rbac.authorization.k8s.io
    Kind: User
    Name: brian
Status: null
```

2. [Space limit](https://raw.githubusercontent.com/kiosk-sh/kiosk/master/examples/account-space-limit.yaml)
3. [Account quotas](https://raw.githubusercontent.com/kiosk-sh/kiosk/master/examples/accountquota.yaml)

---
##### Bottom line
Kiosk seems very powerful with the templates concept but documentation (of that part) is not straight forward.
(May be a documentation pull-request?)