Project: Botkube (Observability and Analysis / Observability)
File: 2022-06-14-policies.md

Created on 2022-06-14 by Mateusz Szostok () This is the alternative solution to address both the configuration design issues and multichannel support. It was replaced by the proposal. This section describes changes in the configuration syntax. As a result, we enable routing notifications to individual channels and limiting channel access to a specific Namespace. Syntax is not backward compatible. New configuration syntax. ```yaml communications: slack: token: 'SLACKAPITOKEN' notifification: type: short policyBinding: # overrides the default configuration channel: dev policies: development channel: prod policies: production admin channel: admin policies: admin ``` New policy syntax. It has refined notification and executors properties. ```yaml policies: name: development executors: kubectl: namespaces include: ns1 ns2 commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] name: network-errors notifications: kubernetes: namespace: include: [".*"] resources: name: v1/pods namespaces: # override top level namespace include: istio-system events: error name: v1/services events: error name: networking.istio.io/v1alpha3/DestinationRules events: error name: networking.istio.io/v1alpha3/VirtualServices events: error ``` The `policy` name can be also changed to `profile`, or `preset` or others. Later, we can create catalog of predefined policies that can be consumed by end-users. This section describes example configurations that enable the requested use-cases. Communicators ```yaml communications: slack: enabled: true token: 'SLACKAPITOKEN' policyBinding: # overrides the default configuration channel: nodes policies: nodes-errors channel: network policies: network-errors ``` Policies ```yaml policies: name: nodes-errors notifications: kubernetes: resources: name: v1/nodes namespaces: include: ".*" events: error name: network-errors notifications: kubernetes: namespace: include: [".*"] resources: name: v1/pods namespaces: # override top level namespace include: istio-system events: error name: v1/services events: error name: networking.istio.io/v1alpha3/DestinationRules events: error name: networking.istio.io/v1alpha3/VirtualServices events: error ``` Communicators ```yaml communications: slack: enabled: true token: 'SLACKAPITOKEN' policyBinding: # overrides the default configuration channel: nodes policies: [nodes-errors, kubectl-full-access] # policies are merged together channel: all policies: # policies are merged together network-errors nodes-errors kubectl-full-access elasticsearch: enabled: true server: 'ELASTICSEARCH_ADDRESS' username: 'ELASTICSEARCH_USERNAME' password: 'ELASTICSEARCH_PASSWORD' policyBinding: # overrides the default configuration index: name: network-errors type: botkube-event shards: 1 replicas: 0 policies: network-errors # the kubectl property is ignored for elasticsearch ``` Policies ```yaml policies: name: nodes-errors notifications: kubernetes: resources: name: v1/nodes namespaces: include: ".*" events: error name: network-errors notifications: kubernetes: namespaces: include: ".*" resources: name: v1/services events: error name: networking.k8s.io/v1/ingresses events: error name: kubectl-full-access executors: kubectl: namespaces: ".*" commands: verbs: ["get", "...", "logs"] resources: ["Deployments", "Pods", "Services"] ``` Currently, you can send notification to non-default channel using . However, you need to apply `botkube.io/channel: <channel_name>` to each K8s object (Pods, Services, etc.) which is cumbersome. With presented configuration you can do that via the `policyBinding`

Communicators ```yaml communications: slack: enabled: true token: 'SLACKAPITOKEN' policyBinding: channel: dev-team-a policies: team-a channel: dev-team-b policies: team-b channel: admin policies: admin ``` Policies ```yaml policies: name: team-a executors: kubectl: namespaces: team-a commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] notifications: kubernetes: namespaces: include: team-a resources: name: v1/pods events: create delete error name: team-b executors: kubectl: namespaces: team-b commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] notifications: kubernetes: namespaces: include: team-b resources: name: v1/pods events: create delete error ``` Communicators ```yaml communications: slack: enabled: true token: 'SLACKAPITOKEN' policyBinding: channel: dev policies: kubectl-full-access channel: team-a policies: kubectl-team-a-ns-access ``` Policies ```yaml policies: name: kubectl-full-access executors: kubectl: enabled: true namespaces: ".*" commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] name: kubectl-team-a-ns-access executors: kubectl: namespaces: team-a commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] ``` If you use the `botkube.io/channel: <channel_name>` annotation, notifications are sent to a given channel even if not authorized. IMO, it's a bug. We can now check if there is a matching `communications.{name}.policyBinding.{channel}` and if policy allows to send such events based on `policies.[.name == foo].notifications`. Other approaches that I consider with explanation why I ruled them out. <details> <summary>Discarded alternative</summary> In the proposed solution, the namespace property is defined separately for executors.kubectl and notifications.resources. In the future it can be added to other executors, e.g. executors.helm. This approach provides fine-grained configuration. You can specify allowed namespace independently, so you can watch for events in all Namespaces but allow `kubectl` usage only in `dev` Namespace. Unfortunately it doesn't come without any cost. If you want to have a strict policy that Team A can access only the `team-a` Namespace, you need to configure that for each executor and notificator. It may be error-prone. To solve that we can extract the namespace property to top level. In this case it will be common for all executors and notificators: ```yaml policies: name: development namespaces: include: team-b kube-system executors: kubectl: enabled: true commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] notifications: resources: name: v1/pods events: create delete error ``` The other option is to extract it even higher. In that way the policies will become a generic description that can be attached to a given platform channel with a given Namespace: ```yaml communications: slack: enabled: true token: 'SLACKAPITOKEN' policyBinding: channel: team-b namespaces: include: team-b policies: example-policy channel: team-a namespaces: include: team-a policies: example-policy ``` ```yaml policies: name: example-policy executors: kubectl: enabled: true commands: verbs: ["get", "logs"] resources: ["Deployments", "Pods", "Services"] notifications: resources: name: v1/pods events: create delete error ``` </details> . Unfortunately, this pull request is too outdated, and the work would need to be started from the ground. Additionally, it doesn't address the syntax issues.