Repository: plexsystems/konstraint
Pull request #79 (2020-10-31 20:39:33)

Add support for parameterized policies
This adds initial support for policies that use Gatekeeper's input parameter functionality. In this implementation, we skip the generation of the `Constraint` resources for policies with parameters, assuming the users will manually create these resources separately. 

Parsing the parameter names from the rules body currently uses regex to avoid diving too deep into the Rego AST package and actually parsing the rules.

Edit: This addresses #78 


Patch:
diff --git a/README.md b/README.md
index 86a7cf6..409c694 100644
--- a/README.md
+++ b/README.md
@@ -58,3 +58,9 @@ By first validating the Kubernetes manifests with `Conftest` on a local machine,
 This typically means no policies were found, or the policies did not have any `violation[]` rules, so they are not compatible with Gatekeeper.
 
 For more information, see [How Constraints are Created](docs/constraint_creation.md).
+
+**My ConstraintTemplates are missing the input parameters**
+
+Input parameters can be specified by using one or more `@parameter <name> <type>` tags in the comment header block. If you use input parameters, Konstraint will skip generating the `Constraint` resource for that policy.
+
+For more information, see [Using Input Parameters](docs/constraint_creation.md#using-input-parameters).
diff --git a/docs/cli/konstraint.md b/docs/cli/konstraint.md
index 5e3a1e7..78c941d 100644
--- a/docs/cli/konstraint.md
+++ b/docs/cli/konstraint.md
@@ -17,4 +17,4 @@ A tool to create and manage Gatekeeper CRDs from Rego
 * [konstraint create](konstraint_create.md)	 - Create Gatekeeper constraints from Rego policies
 * [konstraint doc](konstraint_doc.md)	 - Generate documentation from Rego policies
 
-###### Auto generated by spf13/cobra on 10-Oct-2020
+###### Auto generated by spf13/cobra on 31-Oct-2020
diff --git a/docs/cli/konstraint_create.md b/docs/cli/konstraint_create.md
index e977c8f..fcb63f7 100644
--- a/docs/cli/konstraint_create.md
+++ b/docs/cli/konstraint_create.md
@@ -35,4 +35,4 @@ Create constraints with the Gatekeeper enforcement action set to dryrun
 
 * [konstraint](konstraint.md)	 - Konstraint
 
-###### Auto generated by spf13/cobra on 10-Oct-2020
+###### Auto generated by spf13/cobra on 31-Oct-2020
diff --git a/docs/cli/konstraint_doc.md b/docs/cli/konstraint_doc.md
index 75792b4..05c025e 100644
--- a/docs/cli/konstraint_doc.md
+++ b/docs/cli/konstraint_doc.md
@@ -35,4 +35,4 @@ Set the URL where the policies are hosted at
 
 * [konstraint](konstraint.md)	 - Konstraint
 
-###### Auto generated by spf13/cobra on 10-Oct-2020
+###### Auto generated by spf13/cobra on 31-Oct-2020
diff --git a/docs/constraint_creation.md b/docs/constraint_creation.md
index 18695a4..f1d6a0a 100644
--- a/docs/constraint_creation.md
+++ b/docs/constraint_creation.md
@@ -75,3 +75,33 @@ pod_has_hostipc {
 ```
 
 The comment block is also what is used when generating documentation via the `doc` command.
+
+## Using Input Parameters
+
+Gatekeeper has the ability for a single `ConstraintTemplate` resource to be used by multiple `Constraint`s. One of the reasons for this is that it allows for passing input parameters to the policy so a single policy to avoid duplication. Konstraint supports these input parameters via `@parameter` tags in the header comment block. **NOTE:** When input parameters are specified, Konstraint skips the generation of the `Constraint` resource.
+
+To use parameters, add one or more `@parameter <name> <type>` statements where `<name>` is the name of the parameter and `<type>` is the OpenAPI v3 type of the parameter (string, integer, etc.). Arrays are supported via `@parameter <name> array <type>`. Each parameter tag must be on its own line. Each parameter in the rule body must have a `@parameter` tag in the comment block header.
+
+```rego
+# @title Required Labels
+#
+# This policy allows you to require certain labels are set on a resource.
+#
+# @parameter labels array string
+package required_labels
+
+import data.lib.core
+
+violation[msg] {
+    missing := missing_labels
+    count(missing) > 0
+
+    msg := core.format(sprintf("%s/%s: Missing required labels: %v", [core.kind, core.name, missing]))
+}
+
+missing_labels = missing {
+    provided := {label | core.labels[label]}
+    required := {label | label := input.parameters.labels[_]}
+    missing := required - provided
+}
+```
diff --git a/examples/policies.md b/examples/policies.md
index 87f9f09..64c8721 100755
--- a/examples/policies.md
+++ b/examples/policies.md
@@ -19,6 +19,7 @@
 * [P1013: PodSecurityPolicies must not allow access to the host network](#p1013-podsecuritypolicies-must-not-allow-access-to-the-host-network)
 * [P1014: PodSecurityPolicies must not allow access to the host PID namespace](#p1014-podsecuritypolicies-must-not-allow-access-to-the-host-pid-namespace)
 * [P1015: PodSecurityPolicies must require containers to not run as privileged](#p1015-podsecuritypolicies-must-require-containers-to-not-run-as-privileged)
+* [P0002: Required Labels](#p0002-required-labels)
 * [P2005: Roles must not allow use of privileged PodSecurityPolicies](#p2005-roles-must-not-allow-use-of-privileged-podsecuritypolicies)
 
 ## Warnings
@@ -607,6 +608,45 @@ psp_allows_privileged {
 
 _source: [psp-deny-privileged](psp-deny-privileged)_
 
+## P0002: Required Labels
+
+**Severity:** Violation
+
+**Resources:** Any Resource
+
+**Parameters:**
+
+* labels: array of string
+
+
+This policy allows you to require certain labels are set on a resource.
+Adapted from https://github.com/open-policy-agent/gatekeeper/blob/master/example/templates/k8srequiredlabels_template.yaml
+
+### Rego
+
+```rego
+package required_labels
+
+import data.lib.core
+
+policyID := "P0002"
+
+violation[msg] {
+    missing := missing_labels
+    count(missing) > 0
+
+    msg := core.format_with_id(sprintf("%s/%s: Missing required labels: %v", [core.kind, core.name, missing]), policyID)
+}
+
+missing_labels = missing {
+    provided := {label | core.labels[label]}
+    required := {label | label := input.parameters.labels[_]}
+    missing := required - provided
+}
+```
+
+_source: [required-labels](required-labels)_
+
 ## P2005: Roles must not allow use of privileged PodSecurityPolicies
 
 **Severity:** Violation
diff --git a/examples/required-labels/src.rego b/examples/required-labels/src.rego
new file mode 100644
index 0000000..75b226b
--- /dev/null
+++ b/examples/required-labels/src.rego
@@ -0,0 +1,25 @@
+# @title Required Labels
+#
+# This policy allows you to require certain labels are set on a resource.
+# Adapted from https://github.com/open-policy-agent/gatekeeper/blob/master/example/templates/k8srequiredlabels_template.yaml
+#
+# @parameter labels array string
+
+package required_labels
+
+import data.lib.core
+
+policyID := "P0002"
+
+violation[msg] {
+    missing := missing_labels
+    count(missing) > 0
+
+    msg := core.format_with_id(sprintf("%s/%s: Missing required labels: %v", [core.kind, core.name, missing]), policyID)
+}
+
+missing_labels = missing {
+    provided := {label | core.labels[label]}
+    required := {label | label := input.parameters.labels[_]}
+    missing := required - provided
+}
diff --git a/examples/required-labels/template.yaml b/examples/required-labels/template.yaml
new file mode 100755
index 0000000..667c6e3
--- /dev/null
+++ b/examples/required-labels/template.yaml
@@ -0,0 +1,86 @@
+apiVersion: templates.gatekeeper.sh/v1beta1
+kind: ConstraintTemplate
+metadata:
+  creationTimestamp: null
+  name: requiredlabels
+spec:
+  crd:
+    spec:
+      names:
+        kind: RequiredLabels
+      validation:
+        openAPIV3Schema:
+          properties:
+            labels:
+              items:
+                type: string
+              type: array
+  targets:
+  - libs:
+    - |-
+      package lib.core
+
+      default is_gatekeeper = false
+
+      is_gatekeeper {
+          has_field(input, "review")
+          has_field(input.review, "object")
+      }
+
+      resource = input.review.object {
+          is_gatekeeper
+      }
+
+      resource = input {
+          not is_gatekeeper
+      }
+
+      format(msg) = {"msg": msg} {
+          true
+      }
+
+      format_with_id(msg, id) = msg_fmt {
+          msg_fmt := {
+              "msg": sprintf("%s: %s", [id, msg]),
+              "details": {"policyID": id}
+          }
+      }
+
+      apiVersion = resource.apiVersion
+      name = resource.metadata.name
+      kind = resource.kind
+      labels = resource.metadata.labels
+      annotations = resource.metadata.annotations
+
+      has_field(obj, field) {
+          not object.get(obj, field, "N_DEFINED") == "N_DEFINED"
+      }
+
+      missing_field(obj, field) = true {
+          obj[field] == ""
+      }
+
+      missing_field(obj, field) = true {
+          not has_field(obj, field)
+      }
+    rego: |-
+      package required_labels
+
+      import data.lib.core
+
+      policyID := "P0002"
+
+      violation[msg] {
+          missing := missing_labels
+          count(missing) > 0
+
+          msg := core.format_with_id(sprintf("%s/%s: Missing required labels: %v", [core.kind, core.name, missing]), policyID)
+      }
+
+      missing_labels = missing {
+          provided := {label | core.labels[label]}
+          required := {label | label := input.parameters.labels[_]}
+          missing := required - provided
+      }
+    target: admission.k8s.gatekeeper.sh
+status: {}
diff --git a/go.mod b/go.mod
index 5ae4d17..20e222e 100644
--- a/go.mod
+++ b/go.mod
@@ -8,5 +8,6 @@ require (
 	github.com/open-policy-agent/opa v0.21.0
 	github.com/spf13/cobra v1.0.0
 	github.com/spf13/viper v1.4.0
+	k8s.io/apiextensions-apiserver v0.17.2
 	k8s.io/apimachinery v0.18.4
 )
diff --git a/internal/commands/create.go b/internal/commands/create.go
index bf5bbd0..4bc8301 100644
--- a/internal/commands/create.go
+++ b/internal/commands/create.go
@@ -12,6 +12,7 @@ import (
 	"github.com/open-policy-agent/frameworks/constraint/pkg/apis/templates/v1beta1"
 	"github.com/spf13/cobra"
 	"github.com/spf13/viper"
+	apiextensionsv1beta1 "k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
 	"k8s.io/apimachinery/pkg/runtime/schema"
@@ -84,6 +85,11 @@ func runCreateCommand(path string) error {
 			return fmt.Errorf("writing template: %w", err)
 		}
 
+		// skip Constraint generation if there are parameters on the template
+		if len(violation.Parameters()) > 0 {
+			continue
+		}
+
 		constraint, err := getConstraint(violation)
 		if err != nil {
 			return fmt.Errorf("get constraint: %w", err)
@@ -129,9 +135,35 @@ func getConstraintTemplate(violation rego.Rego) v1beta1.ConstraintTemplate {
 		},
 	}
 
+	if len(violation.Parameters()) > 0 {
+		constraintTemplate.Spec.CRD.Spec.Validation = &v1beta1.Validation{
+			OpenAPIV3Schema: &apiextensionsv1beta1.JSONSchemaProps{
+				Properties: getOpenAPISchemaProperties(violation),
+			},
+		}
+	}
+
 	return constraintTemplate
 }
 
+func getOpenAPISchemaProperties(r rego.Rego) map[string]apiextensionsv1beta1.JSONSchemaProps {
+	properties := make(map[string]apiextensionsv1beta1.JSONSchemaProps)
+	for _, p := range r.Parameters() {
+		if p.IsArray {
+			properties[p.Name] = apiextensionsv1beta1.JSONSchemaProps{
+				Type: "array",
+				Items: &apiextensionsv1beta1.JSONSchemaPropsOrArray{
+					Schema: &apiextensionsv1beta1.JSONSchemaProps{Type: p.Type},
+				},
+			}
+		} else {
+			properties[p.Name] = apiextensionsv1beta1.JSONSchemaProps{Type: p.Type}
+		}
+	}
+
+	return properties
+}
+
 func getConstraint(violation rego.Rego) (unstructured.Unstructured, error) {
 	constraint := unstructured.Unstructured{}
 
diff --git a/internal/commands/document.go b/internal/commands/document.go
index 0141d86..ceb20ff 100644
--- a/internal/commands/document.go
+++ b/internal/commands/document.go
@@ -19,6 +19,7 @@ type Header struct {
 	Description string
 	Resources   string
 	Anchor      string
+	Parameters  []rego.Parameter
 }
 
 // Document is a single policy document.
@@ -132,11 +133,17 @@ func getDocumentation(path string, outputDirectory string) (map[rego.Severity][]
 		anchor := strings.ToLower(strings.ReplaceAll(documentTitle, " ", "-"))
 		anchor = strings.ReplaceAll(anchor, ":", "")
 
+		matchers := policy.Matchers().String()
+		if matchers == "" {
+			matchers = "Any Resource"
+		}
+
 		header := Header{
 			Title:       documentTitle,
 			Description: policy.Description(),
-			Resources:   policy.Matchers().String(),
+			Resources:   matchers,
 			Anchor:      anchor,
+			Parameters:  policy.Parameters(),
 		}
 
 		document := Document{
diff --git a/internal/commands/document_template.go b/internal/commands/document_template.go
index 1dcb34c..e3d3707 100644
--- a/internal/commands/document_template.go
+++ b/internal/commands/document_template.go
@@ -17,6 +17,14 @@ const docTemplate = `# Policies
 
 **Resources:** {{ .Header.Resources }}
 
+{{- if .Header.Parameters }}
+
+**Parameters:**
+
+{{ range .Header.Parameters }}* {{ .Name }}: {{ if .IsArray }}array of {{ end }}{{ .Type }}
+{{ end }}
+{{- end }}
+
 {{ .Header.Description }}
 
 ### Rego
diff --git a/internal/rego/rego.go b/internal/rego/rego.go
index 0c66e12..1d66082 100644
--- a/internal/rego/rego.go
+++ b/internal/rego/rego.go
@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"os"
 	"path/filepath"
+	"regexp"
 	"sort"
 	"strings"
 
@@ -32,6 +33,14 @@ type Rego struct {
 	comments     []string
 	rules        []string
 	dependencies []string
+	parameters   []Parameter
+}
+
+// Parameter represents a parameter that the policy uses
+type Parameter struct {
+	Name    string
+	Type    string
+	IsArray bool
 }
 
 // GetAllSeverities gets all of the rego files found in the given
@@ -81,6 +90,11 @@ func (r Rego) Path() string {
 	return r.path
 }
 
+// Parameters returns the list of parsed parameters
+func (r Rego) Parameters() []Parameter {
+	return r.parameters
+}
+
 // Severity returns the severity of the rego file.
 // When a rego file has multiple rules that are considered
 // to be different severities, the first rule is chosen.
@@ -240,6 +254,28 @@ func parseDirectory(directory string) ([]Rego, error) {
 			comments = append(comments, trimString(string(file.Parsed.Comments[c].Text)))
 		}
 
+		bodyParams := getBodyParamNames(file.Parsed.Rules)
+		headerParams, err := getHeaderParams(comments)
+		if err != nil {
+			return nil, fmt.Errorf("parse header parameters: %w", err)
+		}
+		if len(bodyParams) != len(headerParams) {
+			return nil, fmt.Errorf("count of @parameter tags does not match parameter count")
+		}
+		for _, bodyParam := range bodyParams {
+			var seen bool
+			for _, headerParam := range headerParams {
+				if headerParam.Name == bodyParam {
+					seen = true
+					break
+				}
+			}
+
+			if !seen {
+				return nil, fmt.Errorf("missing @parameter tag for parameter: %s", bodyParam)
+			}
+		}
+
 		// Many YAML parsers do not like rendering out CRLF when writing the YAML to disk.
 		// This causes ConstraintTemplates to be rendered with the line breaks as text,
 		// rather than the actual line break.
@@ -250,6 +286,7 @@ func parseDirectory(directory string) ([]Rego, error) {
 			path:         file.Name,
 			dependencies: dependencies,
 			rules:        rules,
+			parameters:   headerParams,
 			comments:     comments,
 			raw:          raw,
 		}
@@ -264,6 +301,51 @@ func parseDirectory(directory string) ([]Rego, error) {
 	return regos, nil
 }
 
+func getBodyParamNames(rules []*ast.Rule) []string {
+	r := regexp.MustCompile(`input\.parameters\.([a-zA-Z0-9_-]+)`)
+	var bodyParams []string
+	for _, rule := range rules {
+		matches := r.FindAllStringSubmatch(rule.Body.String(), -1)
+		for _, match := range matches {
+			bodyParams = append(bodyParams, match[1]) // the 0 index is the full match, we only care about the first group
+		}
+	}
+	bodyParams = dedupe(bodyParams) // possible a param is referenced more than once
+
+	return bodyParams
+}
+
+func getHeaderParams(comments []string) ([]Parameter, error) {
+	var parameters []Parameter
+	for _, comment := range comments {
+		if strings.HasPrefix(comment, "@parameter") {
+			params := strings.SplitAfter(comment, "@parameter ")[1]
+			paramsSplit := strings.Split(params, " ")
+			if len(paramsSplit) == 0 {
+				return nil, fmt.Errorf("parameter name and type must be specified")
+			}
+			if len(paramsSplit) == 1 {
+				return nil, fmt.Errorf("type must be supplied with parameter name: %s", paramsSplit[0])
+			}
+
+			p := Parameter{Name: paramsSplit[0]}
+			if paramsSplit[1] == "array" {
+				if len(paramsSplit) == 2 {
+					return nil, fmt.Errorf("array type must be supplied with parameter name: %s", paramsSplit[0])
+				}
+				p.IsArray = true
+				p.Type = paramsSplit[2]
+			} else {
+				p.Type = paramsSplit[1]
+			}
+
+			parameters = append(parameters, p)
+		}
+	}
+
+	return parameters, nil
+}
+
 func removeComments(raw string) string {
 	var regoWithoutComments string
 	lines := strings.Split(raw, "\n")


Test patch:
diff --git a/examples/required-labels/src_test.rego b/examples/required-labels/src_test.rego
new file mode 100644
index 0000000..de16c80
--- /dev/null
+++ b/examples/required-labels/src_test.rego
@@ -0,0 +1,30 @@
+package required_labels
+
+test_not_missing {
+    in := {
+        "metadata": {
+            "labels": {
+                "test": "test"
+            }
+        }
+    }
+
+    missing := missing_labels with input as in
+    count(missing) == 0
+}
+
+test_missing {
+    in := {
+        "metadata": {
+            "labels": {
+                "test": "test"
+            }
+        },
+        "parameters": {
+            "labels": ["one", "two"]
+        }
+    }
+
+    missing := missing_labels with input as in
+    count(missing) == 2
+}
diff --git a/internal/rego/rego_test.go b/internal/rego/rego_test.go
index e4b1e52..f3a598f 100644
--- a/internal/rego/rego_test.go
+++ b/internal/rego/rego_test.go
@@ -1,6 +1,7 @@
 package rego
 
 import (
+	"reflect"
 	"testing"
 
 	"github.com/open-policy-agent/opa/ast"
@@ -160,3 +161,54 @@ func TestGetPolicyID_Null(t *testing.T) {
 		t.Errorf("unexpected policyID. expected %v, actual %v", expected, actual)
 	}
 }
+
+func TestGetBodyParamNames(t *testing.T) {
+	ruleString := `violation[msg] {
+	foo := "bar"
+	bar := input.parameters.baz
+	baz := input.parameters.foobars[_]
+	box := input.parameters.baz
+}`
+
+	rule, err := ast.ParseRule(ruleString)
+	if err != nil {
+		t.Fatalf("parse rule: %s", err)
+	}
+
+	expected := []string{"baz", "foobars"}
+	actual := getBodyParamNames([]*ast.Rule{rule})
+	if !(reflect.DeepEqual(expected, actual)) {
+		t.Errorf("unexpected bodyParams. expected %+v, actual %+v", expected, actual)
+	}
+}
+
+func TestGetHeaderParams(t *testing.T) {
+	comments := []string{
+		"@title Title",
+		"Description",
+		"@parameter foo string",
+		"@parameter bar array string",
+		"@kinds another/thing",
+	}
+
+	expected := []Parameter{
+		{
+			Name: "foo",
+			Type: "string",
+		},
+		{
+			Name:    "bar",
+			Type:    "string",
+			IsArray: true,
+		},
+	}
+
+	actual, err := getHeaderParams(comments)
+	if err != nil {
+		t.Fatalf("get header params: %s", err)
+	}
+
+	if !(reflect.DeepEqual(expected, actual)) {
+		t.Errorf("unexpected headerParams. expected %+v, actual %+v", expected, actual)
+	}
+}