Repository: redhat-certification/chart-verifier (go)

Issue:
Running IBM chart ibm-object-storage-plugin-2.0.8.tgz fails with License issue
We (@mmulholla and I) were working with one of the IBMers trying to run the chart verifier. This is the feedback she gave us:

1. Expose the error so that the user knows what is wrong with the chart testing in the local cluster.
2. Honor the license value in the rendered templates. See the error below with older version of chart verifier showing the errors:
3. Add more example runs that pass in the docs so that developers can compare with their environments. 

You can get their charts here: https://github.com/IBM/charts/blob/master/repo/ibm-helm/ibm-object-storage-plugin-2.0.8.tgz

dperaza@dperaza-mac Downloads % chart-verifier verify --chart-set license=true $HOME/Downloads/ibm-object-storage-plugin-2.0.8.tgz 2>&1
Error: execution error at (ibm-object-storage-plugin/templates/flex-driver-sa.yaml:3:61): License must be accepted by setting license key to true
Deleting release 'ibm-object-storage-plugin-mng7o4fjcq'...
Error: uninstall: Release not loaded: ibm-object-storage-plugin-mng7o4fjcq: release: not found
Error deleting Helm release: Error waiting for process: exit status 1
apiversion: v1
kind: verify-report
metadata:
    tool:
        verifier-version: 1.0.0
        chart-uri: /Users/dperaza/Downloads/ibm-object-storage-plugin-2.0.8.tgz
        digest: sha256:71334727f9bf10ece92c8d79eebac55a4d78cd4b21b63f0a9555310207004591
        lastCertifiedTimestamp: "2021-05-24T16:24:00.879823-04:00"
        certifiedOpenShiftVersions: 4.7.5
    chart:
        name: ibm-object-storage-plugin
        home: ""
        sources: []
        version: 2.0.8
        description: |
            Chart for deploying ibmcloud object storage plugin.
            IBM Cloud Object Storage is persistent, highly available storage
            that you can mount to apps that run in a Kubernetes cluster by using
            the IBM Cloud Object Storage plug-in. The plug-in is a Kubernetes Flex-Volume plug-in
            that connects Cloud Object Storage buckets to pods in your cluster.
        keywords:
          - IKS
          - amd64
          - Storage
          - Commercial
          - Limited
          - Other
          - ROKS
          - ICP
        maintainers:
          - name: IBM
            email: contsto2@in.ibm.com
            url: ""
        icon: https://cache.globalcatalog.cloud.ibm.com/static/cache/2461-acb0d10a1725d783/images/uploaded/icons/object-storage.png
        apiversion: v2
        condition: ""
        tags: ""
        appversion: 2.0.8
        deprecated: false
        annotations: {}
        kubeversion: '>=1.10.1-0'
        dependencies: []
        type: ""
    overrides: ""
results:
  - check: not-contain-csi-objects
    type: Mandatory
    outcome: PASS
    reason: CSI objects do not exist
  - check: chart-testing
    type: Mandatory
    outcome: FAIL
    reason: 'Error waiting for process: exit status 1'
  - check: has-readme
    type: Mandatory
    outcome: PASS
    reason: Chart has a README
  - check: contains-test
    type: Mandatory
    outcome: PASS
    reason: Chart test files exist
  - check: contains-values-schema
    type: Mandatory
    outcome: PASS
    reason: Values schema file exist
  - check: not-contains-crds
    type: Mandatory
    outcome: PASS
    reason: Chart does not contain CRDs
  - check: images-are-certified
    type: Mandatory
    outcome: PASS
    reason: |-
        Image is Red Hat certified : ibmcom/ibmcloud-object-storage-driver@sha256:e07cf3ff8d56663fc33d19f0294a0c3f969038695a1d98f60fb762cbb6324053
        Image is Red Hat certified : ibmcom/ibmcloud-object-storage-plugin@sha256:bddac7c558cea2379724360b6a1fe8a50779d093d87047e16c0f967e2c7224a0
  - check: is-helm-v3
    type: Mandatory
    outcome: PASS
    reason: API version is V2, used in Helm 3
  - check: contains-values
    type: Mandatory
    outcome: PASS
    reason: Values file exist
  - check: has-kubeversion
    type: Mandatory
    outcome: PASS
    reason: Kubernetes version specified
  - check: helm-lint
    type: Mandatory
    outcome: PASS
    reason: Helm lint successful

Pull request description:
Make chart-testing check honor values informed through `--chart-set`
This change list adjusts the `chart-testing` check to propagate the
values informed by the user through `--chart-set` and friends by
merging those values into the `values.yaml` file for the test context.

- [x] Basic implementation
- [x] Make sure `--chart-set` values are propagated to `chart-testing`
      check.
- [ ] Skip `chart-testing` tests in the case the file the `KUBECONFIG`
      environment variable is referring to does not exist.
	  
Closes #134.

Patch:
diff --git a/go.mod b/go.mod
index b324673..96bf9a3 100644
--- a/go.mod
+++ b/go.mod
@@ -6,6 +6,7 @@ require (
 	github.com/Masterminds/semver v1.5.0
 	github.com/hashicorp/go-retryablehttp v0.6.8 // indirect
 	github.com/helm/chart-testing/v3 v3.3.1
+	github.com/imdario/mergo v0.3.11
 	github.com/mitchellh/go-homedir v1.1.0
 	github.com/pkg/errors v0.9.1
 	github.com/spf13/cobra v1.1.1
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/.helmignore b/pkg/chartverifier/checks/psql-service-0.1.7/.helmignore
new file mode 100644
index 0000000..0e8a0eb
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/.helmignore
@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/Chart.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/Chart.yaml
new file mode 100644
index 0000000..2278598
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/Chart.yaml
@@ -0,0 +1,12 @@
+annotations:
+  charts.openshift.io/archs: x86_64
+  charts.openshift.io/name: PSQL RedHat Demo Chart
+  charts.openshift.io/provider: RedHat
+  charts.openshift.io/supportURL: https://github.com/dperaza4dustbit/helm-chart
+apiVersion: v2
+appVersion: 10.0.0
+description: A Helm chart for a RedHat Certified PSQL
+kubeVersion: 1.20.0
+name: psql-service
+type: application
+version: 0.1.7
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/README.md b/pkg/chartverifier/checks/psql-service-0.1.7/README.md
new file mode 100644
index 0000000..17ba700
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/README.md
@@ -0,0 +1,1 @@
+Helm chart for psql 10 certified image
\ No newline at end of file
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/runtime_value.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/runtime_value.yaml
new file mode 100644
index 0000000..6f8d560
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/runtime_value.yaml
@@ -0,0 +1,5 @@
+serviceAccount:
+  enabled: true
+  name: sa-with-anyuid
+
+postgresqlPassword: quebolasere
\ No newline at end of file
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/templates/_helpers.tpl b/pkg/chartverifier/checks/psql-service-0.1.7/templates/_helpers.tpl
new file mode 100644
index 0000000..233e3f4
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/templates/_helpers.tpl
@@ -0,0 +1,62 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "ssm-service.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "ssm-service.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "ssm-service.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "ssm-service.labels" -}}
+helm.sh/chart: {{ include "ssm-service.chart" . }}
+{{ include "ssm-service.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "ssm-service.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "ssm-service.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "ssm-service.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "ssm-service.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/templates/deployment.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/templates/deployment.yaml
new file mode 100644
index 0000000..291983c
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/templates/deployment.yaml
@@ -0,0 +1,53 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ .Values.k8Name }}
+  namespace: {{ .Values.k8Project }}
+  annotations:
+      app.kubernetes.io/part-of: ssm
+spec:
+  selector:
+    matchLabels: 
+      app: {{ .Values.k8Name }}
+  replicas: {{ .Values.replicaCount }}
+  template:
+    metadata:
+      annotations:
+        alpha.image.policy.openshift.io/resolve-names: '*'
+      labels:
+        app: {{ .Values.k8Name }}
+        phase: test
+    spec:
+      # securityContext:
+      #   runAsUser: {{ .Values.securityContext.runAsUser }}
+      #   fsGroup: {{ .Values.podSecurityContext.fsGroup }}
+      # serviceAccount: {{ .Values.serviceAccount }}
+      containers:
+      - name: {{ .Values.k8Name }}
+        image: '{{ .Values.image.repository }}{{ .Values.image.name }}:{{ .Values.image.tag }}'
+        env:
+        - name: POSTGRESQL_USER
+          value: {{ .Values.config.postgresUser }}
+        - name: POSTGRESQL_PASSWORD
+          value: {{ .Values.config.postgresPassword }}
+        - name: POSTGRESQL_DATABASE
+          value: {{ .Values.config.postgresDatabase }}
+        - name: CHART_VERSION
+          value: {{ .Chart.AppVersion }}
+        ports:
+          - containerPort: {{ .Values.service.port }}
+        securityContext:
+          runAsNonRoot: true
+        resources:
+          limits:
+            cpu: {{ .Values.resources.limits.cpu }}
+            memory: {{ .Values.resources.limits.memory }}
+          requests:
+            cpu: {{ .Values.resources.requests.cpu }}
+            memory: {{ .Values.resources.requests.memory }}
+        readinessProbe:
+          tcpSocket:
+            port: {{ .Values.service.port }}
+          initialDelaySeconds: 15
+          periodSeconds: 20
+
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/templates/service.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/templates/service.yaml
new file mode 100644
index 0000000..0b091b0
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/templates/service.yaml
@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ .Values.k8ServiceName }}
+  namespace: {{ .Values.k8Project }}
+spec:
+  type: {{ .Values.service.type }}
+  # loadBalancerSourceRanges:
+  #   - {{ .Values.service.sourceRange }}
+  ports:
+    - name: rest
+      port: {{ .Values.service.port }}
+      targetPort: {{ .Values.service.port }}
+  selector:
+    app: {{ .Values.k8Name }}
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/values.schema.json b/pkg/chartverifier/checks/psql-service-0.1.7/values.schema.json
new file mode 100644
index 0000000..814b7bb
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/values.schema.json
@@ -0,0 +1,59 @@
+{
+    "$schema": "http://json-schema.org/schema#",
+    "type": "object",
+    "required": [
+      "image",
+      "k8Project",
+      "config"
+    ],
+    "properties": {
+      "image": {
+        "type": "object",
+        "required": [
+          "name",
+          "repository",
+          "pullPolicy"
+        ],
+        "properties": {
+          "name": {
+            "type": "string",
+            "pattern": "^[a-z0-9-_]+$"
+          },
+          "repository": {
+            "type": "string",
+            "pattern": "^[a-z0-9-_./]+$"
+          },
+          "pullPolicy": {
+            "type": "string",
+            "pattern": "^(Always|Never|IfNotPresent)$"
+          }
+        }
+      },
+      "k8Project": {
+        "type": "string",
+        "pattern": "^[a-z0-9-_]+$"
+      },
+      "config": {
+        "type": "object",
+        "required": [
+          "postgresUser",
+          "postgresPassword",
+          "postgresDatabase"
+        ],
+        "properties": {
+          "postgresUser": {
+            "type": "string",
+            "pattern": "^[a-z0-9-_]+$"
+          },
+          "postgresPassword": {
+            "type": "string",
+            "pattern": "^[a-z0-9-_]+$"
+          },
+          "postgresDatabase": {
+            "type": "string",
+            "pattern": "^[a-z0-9-_]+$"
+          }
+        }
+      }     
+    }   
+  }
\ No newline at end of file
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/values.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/values.yaml
new file mode 100644
index 0000000..fcfed1d
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/values.yaml
@@ -0,0 +1,42 @@
+# Default values for ssm-service.
+# This is a YAML-formatted file.
+# Declare variables to be passed into your templates.
+
+replicaCount: 1
+k8Project: ""
+k8Name: psql-bee
+k8ServiceName: psql
+
+serviceAccount: sa-with-anyuid
+
+config:
+  postgresUser: dperaza
+  postgresPassword: quebolasere
+  postgresDatabase: helmrocks
+
+image:
+  name: postgresql-10-rhel7
+  repository: "registry.access.redhat.com/rhscl/"
+  pullPolicy: IfNotPresent
+  tag: "latest"
+
+
+podSecurityContext: 
+  fsGroup: 2000
+
+securityContext:
+  runAsUser: 1000
+  
+
+service:
+  type: ClusterIP
+  port: 5432
+  sourceRange: 0.0.0.0/0
+
+resources:
+  limits:
+    cpu: 100m
+    memory: 128Mi
+  requests:
+    cpu: 100m
+    memory: 128Mi
diff --git a/pkg/tool/helm.go b/pkg/tool/helm.go
index bf0c3a7..c9ab01f 100644
--- a/pkg/tool/helm.go
+++ b/pkg/tool/helm.go
@@ -1,6 +1,9 @@
 package tool
 
 import (
+	"fmt"
+	"strings"
+
 	"github.com/helm/chart-testing/v3/pkg/exec"
 	"github.com/helm/chart-testing/v3/pkg/tool"
 )
@@ -27,9 +30,12 @@ func (h Helm) InstallWithValues(chart string, valuesFile string, namespace strin
 		values = []string{"--values", valuesFile}
 	}
 
-	if _, err := h.RunProcessAndCaptureOutput("helm", "install", release, chart, "--namespace", namespace,
-		"--wait", values, h.extraArgs); err != nil {
-		return err
+	helmInstallArgs := []string{"install", release, chart, "--namespace", namespace, "--wait"}
+	helmInstallArgs = append(helmInstallArgs, values...)
+	helmInstallArgs = append(helmInstallArgs, h.extraArgs...)
+
+	if _, err := h.RunProcessAndCaptureOutput("helm", helmInstallArgs); err != nil {
+		return fmt.Errorf("executing helm with args %q: %w", strings.Join(helmInstallArgs, " "), err)
 	}
 
 	return nil


Test patch:
diff --git a/pkg/chartverifier/checks/charttesting.go b/pkg/chartverifier/checks/charttesting.go
index 9277685..034c5e1 100644
--- a/pkg/chartverifier/checks/charttesting.go
+++ b/pkg/chartverifier/checks/charttesting.go
@@ -2,13 +2,18 @@ package checks
 
 import (
 	"fmt"
+	"io/ioutil"
+	"os"
+	"path"
 	"strings"
 
 	"github.com/helm/chart-testing/v3/pkg/chart"
 	"github.com/helm/chart-testing/v3/pkg/config"
 	"github.com/helm/chart-testing/v3/pkg/exec"
 	"github.com/helm/chart-testing/v3/pkg/util"
+	"github.com/imdario/mergo"
 	"github.com/redhat-certification/chart-verifier/pkg/tool"
+	"gopkg.in/yaml.v3"
 )
 
 // buildChartTestingConfiguration computes the chart testing related
@@ -96,7 +101,7 @@ func ChartTesting(opts *CheckOptions) (Result, error) {
 			return NewResult(false, result.Error.Error()), nil
 		}
 	} else {
-		result := installAndTestChartRelease(cfg, chrt, helm, kubectl)
+		result := installAndTestChartRelease(cfg, chrt, helm, kubectl, opts.Values)
 		if result.Error != nil {
 			return NewResult(false, result.Error.Error()), nil
 		}
@@ -213,12 +218,90 @@ func upgradeAndTestChart(
 	return result
 }
 
+// readObjectFromYamlFile unmarshals the given filename and returns an object with its contents.
+func readObjectFromYamlFile(filename string) (map[string]interface{}, error) {
+	objBytes, err := ioutil.ReadFile(filename)
+	if err != nil {
+		return nil, fmt.Errorf("reading values file: %w", err)
+	}
+
+	var obj map[string]interface{}
+	err = yaml.Unmarshal(objBytes, &obj)
+	if err != nil {
+		return nil, fmt.Errorf("unmarshalling values file contents: %w", err)
+	}
+
+	return obj, nil
+}
+
+// writeObjectToTempYamlFile writes the given obj into a temporary file and returns its location,
+//
+// It is responsibility of the caller to discard the file when finished using it.
+func writeObjectToTempYamlFile(obj map[string]interface{}) (filename string, cleanupFunc func(), err error) {
+	objBytes, err := yaml.Marshal(obj)
+	if err != nil {
+		return "", nil, fmt.Errorf("marshalling values file new contents: %w", err)
+	}
+
+	tempDir, err := ioutil.TempDir(os.TempDir(), "chart-testing-*")
+	if err != nil {
+		return "", nil, fmt.Errorf("creating temporary directory: %w", err)
+	}
+
+	filename = path.Join(tempDir, "values.yaml")
+
+	err = ioutil.WriteFile(filename, objBytes, 0644)
+	if err != nil {
+		return "", nil, fmt.Errorf("writing values file new contents: %w", err)
+	}
+
+	cleanupFunc = func() {
+		os.RemoveAll(tempDir)
+	}
+
+	return filename, cleanupFunc, nil
+}
+
+// newTempValuesFileWithOverrides applies the extra values provided into the given filename (a YAML file) and materializes its
+// contents in the file returned by the function.
+//
+// In the case the given filename is an empty string, it indicates that only the valueOverrides contents will be
+// materialized into the temporary file to be merged by `helm` when processing the chart.
+func newTempValuesFileWithOverrides(filename string, valuesOverrides map[string]interface{}) (string, func(), error) {
+
+	var obj map[string]interface{}
+
+	if filename != "" {
+		// in the case a filename is provided, read its contents and merge any available values override.
+		obj, err := readObjectFromYamlFile(filename)
+		if err != nil {
+			return "", nil, fmt.Errorf("reading values file: %w", err)
+		}
+
+		err = mergo.MergeWithOverwrite(obj, valuesOverrides)
+		if err != nil {
+			return "", nil, fmt.Errorf("merging extra values: %w", err)
+		}
+
+	} else {
+		obj = valuesOverrides
+	}
+
+	newValuesFile, clean, err := writeObjectToTempYamlFile(obj)
+	if err != nil {
+		return "", nil, fmt.Errorf("writing object to temporary location: %w", err)
+	}
+
+	return newValuesFile, clean, nil
+}
+
 // installAndTestChartRelease installs and tests a chart release.
 func installAndTestChartRelease(
 	cfg config.Configuration,
 	chrt *chart.Chart,
 	helm tool.Helm,
 	kubectl tool.Kubectl,
+	valuesOverrides map[string]interface{},
 ) chart.TestResult {
 
 	// valuesFiles contains all the configurations that should be
@@ -238,10 +321,19 @@ func installAndTestChartRelease(
 		// Use anonymous function. Otherwise deferred calls would pile up
 		// and be executed in reverse order after the loop.
 		fun := func() error {
-			namespace, release, releaseSelector, cleanup := generateInstallConfig(cfg, chrt, helm, kubectl)
-			defer cleanup()
 
-			if err := helm.InstallWithValues(chrt.Path(), valuesFile, namespace, release); err != nil {
+			tmpValuesFile, tmpValuesFileCleanup, err := newTempValuesFileWithOverrides(valuesFile, valuesOverrides)
+			if err != nil {
+				// it is required this operation to succeed, otherwise there are no guarantees the values informed using
+				// `--chart-set` are propagated to the installation process, so the process breaks here.
+				return fmt.Errorf("creating temporary values file: %w", err)
+			}
+			defer tmpValuesFileCleanup()
+
+			namespace, release, releaseSelector, releaseCleanup := generateInstallConfig(cfg, chrt, helm, kubectl)
+			defer releaseCleanup()
+
+			if err := helm.InstallWithValues(chrt.Path(), tmpValuesFile, namespace, release); err != nil {
 				return err
 			}
 			return testRelease(helm, kubectl, release, namespace, releaseSelector, false)
diff --git a/pkg/chartverifier/checks/charttesting_test.go b/pkg/chartverifier/checks/charttesting_test.go
index a8e5046..901603e 100644
--- a/pkg/chartverifier/checks/charttesting_test.go
+++ b/pkg/chartverifier/checks/charttesting_test.go
@@ -1,6 +1,12 @@
 package checks
 
 import (
+	"fmt"
+	"os"
+	"os/exec"
+	"path"
+	"path/filepath"
+	"runtime"
 	"testing"
 
 	"github.com/spf13/viper"
@@ -8,67 +14,106 @@ import (
 	"helm.sh/helm/v3/pkg/cli"
 )
 
+// absPathFromSourceFileLocation returns the absolute path of a file or directory under the current source file's
+// location.
+func absPathFromSourceFileLocation(name string) (string, error) {
+	_, filename, _, ok := runtime.Caller(1)
+	if !ok {
+		panic("couldn't get current path")
+	}
+	filename, err := filepath.Abs(filename)
+	if err != nil {
+		return "", fmt.Errorf("retrieving current source file's location: %w", err)
+	}
+	dirname := path.Dir(filename)
+	return filepath.Join(dirname, name), nil
+}
+
+func lookPath(programs ...string) error {
+	for _, p := range programs {
+		_, err := exec.LookPath(p)
+		if err != nil {
+			return fmt.Errorf("required program %q not found", p)
+		}
+	}
+	return nil
+}
+
 func TestChartTesting(t *testing.T) {
+	if os.Getenv("CHART_VERIFIER_ENABLE_CLUSTER_TESTING") == "" {
+		t.Skip("CHART_VERIFIER_ENABLE_CLUSTER_TESTING not set, skipping in cluster tests")
+	}
+
+	if err := lookPath("helm", "kubectl"); err != nil {
+		t.Skip(err.Error())
+	}
+
 	type testCase struct {
-		config      map[string]interface{}
 		description string
-		uri         string
+		opts        CheckOptions
 	}
 
-	testCases := []testCase{
-		{
-			config:      map[string]interface{}{},
-			description: "with chart-testing defaults",
-			uri:         "chart-0.1.0-v3.valid.tgz",
-		},
-		{
-			config: map[string]interface{}{
-				"upgrade": true,
-			},
-			description: "override chart-testing upgrade",
-			uri:         "chart-0.1.0-v3.valid.tgz",
-		},
+	chartUri, err := absPathFromSourceFileLocation("psql-service-0.1.7")
+	if err != nil {
+		t.Error(err)
+	}
+
+	positiveTestCases := []testCase{
 		{
-			config: map[string]interface{}{
-				"skipMissingValues": true,
+			description: "providing a valid k8Project value should succeed",
+			opts: CheckOptions{
+				URI: chartUri,
+				Values: map[string]interface{}{
+					"k8Project": "default",
+				},
+				ViperConfig:     viper.New(),
+				HelmEnvSettings: cli.New(),
 			},
-			description: "override chart-testing upgrade",
-			uri:         "chart-0.1.0-v3.valid.tgz",
 		},
+	}
+
+	for _, tc := range positiveTestCases {
+		t.Run(tc.description, func(t *testing.T) {
+			r, err := ChartTesting(&tc.opts)
+			require.NoError(t, err)
+			require.NotNil(t, r)
+			require.Equal(t, ChartTestingSuccess, r.Reason)
+			require.True(t, r.Ok)
+		})
+	}
+
+	negativeTestCases := []testCase{
 		{
-			config: map[string]interface{}{
-				"namespace": "ct-test-namespace",
+			description: "providing a bogus k8Project should fail",
+			opts: CheckOptions{
+				URI: chartUri,
+				Values: map[string]interface{}{
+					"k8Project": "bogus",
+				},
+				ViperConfig:     viper.New(),
+				HelmEnvSettings: cli.New(),
 			},
-			description: "override chart-testing namespace",
-			uri:         "chart-0.1.0-v3.valid.tgz",
 		},
 		{
-			config: map[string]interface{}{
-				"releaseLabel": "chart-verifier-app.kubernetes.io/instance",
+			// the chart being used in this test forces the rendered resources to have an empty namespace field, which
+			// is invalid and can't be overriden using helm's namespace option.
+			description: "empty values should fail",
+			opts: CheckOptions{
+				URI:             chartUri,
+				Values:          map[string]interface{}{},
+				ViperConfig:     viper.New(),
+				HelmEnvSettings: cli.New(),
 			},
-			description: "override chart-testing releaseLabel",
-			uri:         "chart-0.1.0-v3.valid.tgz",
 		},
 	}
 
-	for _, tc := range testCases {
-		config := viper.New()
-		settings := cli.New()
-
-		_ = config.MergeConfigMap(tc.config)
-
+	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			t.Skip()
-			r, err := ChartTesting(
-				&CheckOptions{
-					URI:             tc.uri,
-					ViperConfig:     config,
-					HelmEnvSettings: settings,
-				},
-			)
-			require.NoError(t, err)
+			r, err := ChartTesting(&tc.opts)
 			require.NotNil(t, r)
-			require.True(t, r.Ok)
+			require.False(t, r.Ok)
+			require.NoError(t, err)
+			require.Contains(t, r.Reason, "executing helm with args")
 		})
 	}
 }
diff --git a/pkg/chartverifier/checks/psql-service-0.1.7/templates/tests/test-psql-connection.yaml b/pkg/chartverifier/checks/psql-service-0.1.7/templates/tests/test-psql-connection.yaml
new file mode 100644
index 0000000..2779a43
--- /dev/null
+++ b/pkg/chartverifier/checks/psql-service-0.1.7/templates/tests/test-psql-connection.yaml
@@ -0,0 +1,26 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: "{{ .Release.Name }}-credentials-test"
+  #namespace: {{ .Values.k8Project }}
+  annotations:
+    "helm.sh/hook": test-success
+spec:
+  #serviceAccount: {{ .Values.serviceAccount }}
+  containers:
+    - name: {{ .Release.Name }}-credentials-test
+      image: '{{ .Values.image.repository }}{{ .Values.image.name }}:{{ .Values.image.tag }}'
+      imagePullPolicy: {{ .Values.image.pullPolicy | quote }}
+      env:
+        - name: POSTGRESQL_USER
+          value: {{ .Values.config.postgresUser }}
+        - name: PGPASSWORD
+          value: {{ .Values.config.postgresPassword }}
+        - name: POSTGRESQL_DATABASE
+          value: {{ .Values.config.postgresDatabase }}
+      command:
+        - /bin/bash
+        - -ec
+        - |
+          psql -d $POSTGRESQL_DATABASE -h psql -p $PSQL_SERVICE_PORT -U $POSTGRESQL_USER -c "select 1"
+  restartPolicy: Never