Repository: redhat-certification/chart-verifier (go)

Issue:
Propagate user options to checks
It is a fair assumption that users might want to parametrize checks in different situations; one particular example is a check asserting whether the Helm chart is compatible with a given version of OpenShift, which needs the user's input to specify which one.

The `--set` flag should be added to the verify command; its value has the `KEY=VALUE` format, where `KEY` is the path to a particular configuration field, and `VALUE` is its value. For example `verify --set compat.version=openshift-4.6` would overwrite the field `compat.version` if defined in the configuration file. The flag should also be accepted multiple times.

 

Pull request description:
Propagate User Defined Configuration To Checks 
There are multiple use cases where propagating user defined configuration to checks is useful: to specify an OpenShift
version to verify the chart compatibility or influence the severity the Helm linter check should consider a failure; in
other words, any occasion the user needs to parametrize a check.

Closes #23.

## Rationale

`chart-verifier` is the command (either through a Container Runtime or directly using the program directly) users
interface to provide information regarding the verifications to be performed for a given chart, so it is expected that
different parameters can be used at different moments in time.

Those parameters can be given to the program in two ways: through the configuration file, which is already supported by
not yet used; and through command line flags that can be used to overwrite a value defined by the configuration file.

Having both mechanisms to influence the verification session is very useful for a couple of reasons:

1. Configuration files can be distributed as *profiles*, for example one for OpenShift 4.6, another for OpenShift 4.7;
1. Profile defaults can be overridden, helping developers and other power users to debug, change checks parameters
   before committing to write a configuration/profile file.

## Usage

Some changes are required in the `chart-verifier` program, more specifically in the `verify` command, where the
flag `--set` should be introduced, as in the example below:

```text
> chart-verifier verify --set compat.version=openshift-4.6 --enable compat chart.tgz
```

As the example above shows, the format of a `--set` flag is `KEY=VALUE`, where `KEY` is the path of the value in the
configuration file and `VALUE` the value to be assigned to the configuration.

In the example, `compat` is the check name, and `version` is the key the `compat` check will use to verify the
compatibility of the given chart, in this case against the `openshift-4.6` profile.

Another example related to the severity of the Helm linter check:

```text
> chart-verifier verify --set linter.failWhen=ERROR --enable linter chart.tgz
```

Both configurations could be use simultaneously by providing multiple `--set` flags:

```text
> chart-verifier verify                \
    --set compat.version=openshift-4.6 \
    --set linter.failWhen=ERROR        \
    --enable compat,linter             \
    chart.tgz
```

The configuration file could be used to store the same configuration expressed by the usage of the `--set` flag above:

```yaml
compat:
    version: openshift-4.6
linter:
    failWhen: ERROR
```

The example below uses the configuration file above:

```text
> chart-verifier verify --config openshift-4.6.yaml chart.tgz
```

## Checks API

Currently the `CheckFunc` type is defined as the example below:

```go
type CheckFunc func(uri string) (Result, error)
```

The current `CheckFunc` type is too simplistic, and should be modified to either receive an extra parameter containing
the options to be considered by the check, or a more complex type composing both of this values:

```go
type CheckFunc func(uri string, config *viper.Viper) (Result, error)
```

`*viper.Viper` is used since it has already been integrated in the main program, and also because it offers the required configuration semantics to isolate checks configuration.

The `opts` map is configuration for the check found in the configuration file overridden by the `--set`; in the `openshift-4.6.yaml`, the `compat` check would expect the following values in `opts`:

```go
opts := map[string]interface{}{
    "version": "openshift-4.6"
}
```

## Test Scenarios

There are a couple of scenarios that should be tested, in order to validate the proposal implementation (assuming a `dummy` check is available):

* The `dummy` check should fail by default, regardless the chart;
* The `dummy` check should succeed when `dummy.ok` is set to `true` using the `--set` flag;
* The `dummy` check should succeed when the configuration file configures the `dummy.ok` configuration value to `true`; and
* The `dummy` check should fail when the configuration file configures the `dummy.ok` configuration value to `true` and `dummy.ok` is set to `false` using the `--set` flag.


Patch:
diff --git a/cmd/verify.go b/cmd/verify.go
index d9eaa66..91ace24 100644
--- a/cmd/verify.go
+++ b/cmd/verify.go
@@ -22,6 +22,7 @@ import (
 	"github.com/pkg/errors"
 
 	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
 	"gopkg.in/yaml.v3"
 
 	"github.com/redhat-certification/chart-verifier/pkg/chartverifier"
@@ -41,6 +42,8 @@ var (
 	disabledChecksFlag []string
 	// outputFormatFlag contains the output format the user has specified: default, yaml or json.
 	outputFormatFlag string
+	// setOverridesFlag contains the overrides the user has specified through the --set flag.
+	setOverridesFlag []string
 )
 
 func filterChecks(set []string, subset []string, setEnabled bool, subsetEnabled bool) ([]string, error) {
@@ -76,13 +79,8 @@ func buildChecks(all, enabled, disabled []string) ([]string, error) {
 	}
 }
 
-func buildCertifier(checks []string) (chartverifier.Certifier, error) {
-	return chartverifier.NewCertifierBuilder().
-		SetChecks(checks).
-		Build()
-}
-
-func NewVerifyCmd() *cobra.Command {
+// NewVerifyCmd creates ...
+func NewVerifyCmd(config *viper.Viper) *cobra.Command {
 	cmd := &cobra.Command{
 		Use:   "verify <chart-uri>",
 		Args:  cobra.ExactArgs(1),
@@ -93,7 +91,13 @@ func NewVerifyCmd() *cobra.Command {
 				return err
 			}
 
-			certifier, err := buildCertifier(checks)
+			certifier, err := chartverifier.
+				NewCertifierBuilder().
+				SetChecks(checks).
+				SetConfig(config).
+				SetOverrides(setOverridesFlag).
+				Build()
+
 			if err != nil {
 				return err
 			}
@@ -132,12 +136,11 @@ func NewVerifyCmd() *cobra.Command {
 
 	cmd.Flags().StringVarP(&outputFormatFlag, "output", "o", "", "the output format: default, json or yaml")
 
+	cmd.Flags().StringSliceVarP(&setOverridesFlag, "set", "s", []string{}, "overrides a configuration, e.g: dummy.ok=false")
+
 	return cmd
 }
 
-// verifyCmd represents the lint command
-var verifyCmd = NewVerifyCmd()
-
 func init() {
-	rootCmd.AddCommand(verifyCmd)
+	rootCmd.AddCommand(NewVerifyCmd(viper.GetViper()))
 }
diff --git a/pkg/chartverifier/certifier.go b/pkg/chartverifier/certifier.go
index d98d8e1..4a51ca6 100644
--- a/pkg/chartverifier/certifier.go
+++ b/pkg/chartverifier/certifier.go
@@ -18,6 +18,7 @@ package chartverifier
 
 import (
 	"github.com/redhat-certification/chart-verifier/pkg/chartverifier/checks"
+	"github.com/spf13/viper"
 )
 
 type CheckNotFoundErr string
@@ -37,10 +38,19 @@ func NewCheckErr(err error) error {
 }
 
 type certifier struct {
+	config         *viper.Viper
 	registry       checks.Registry
 	requiredChecks []string
 }
 
+func (c *certifier) subConfig(name string) *viper.Viper {
+	if sub := c.config.Sub(name); sub == nil {
+		return viper.New()
+	} else {
+		return sub
+	}
+}
+
 func (c *certifier) Certify(uri string) (Certificate, error) {
 
 	chrt, _, err := checks.LoadChartFromURI(uri)
@@ -53,15 +63,17 @@ func (c *certifier) Certify(uri string) (Certificate, error) {
 		SetChartVersion(chrt.AppVersion())
 
 	for _, name := range c.requiredChecks {
-		if checkFunc, ok := c.registry.Get(name); !ok {
+		checkFunc, ok := c.registry.Get(name)
+		if !ok {
 			return nil, CheckNotFoundErr(name)
-		} else {
-			r, err := checkFunc(uri)
-			if err != nil {
-				return nil, NewCheckErr(err)
-			}
-			_ = result.AddCheckResult(name, r)
 		}
+
+		r, err := checkFunc(uri, c.subConfig(name))
+		if err != nil {
+			return nil, NewCheckErr(err)
+		}
+		_ = result.AddCheckResult(name, r)
+
 	}
 
 	return result.Build()
diff --git a/pkg/chartverifier/certifierbuilder.go b/pkg/chartverifier/certifierbuilder.go
index f1ed234..06dfb21 100644
--- a/pkg/chartverifier/certifierbuilder.go
+++ b/pkg/chartverifier/certifierbuilder.go
@@ -18,6 +18,9 @@ package chartverifier
 
 import (
 	"errors"
+	"strings"
+
+	"github.com/spf13/viper"
 
 	"github.com/redhat-certification/chart-verifier/pkg/chartverifier/checks"
 )
@@ -26,6 +29,7 @@ var defaultRegistry checks.Registry
 
 func init() {
 	defaultRegistry = checks.NewRegistry()
+	defaultRegistry.Add("dummy", checks.Dummy)
 	defaultRegistry.Add("has-readme", checks.HasReadme)
 	defaultRegistry.Add("is-helm-v3", checks.IsHelmV3)
 	defaultRegistry.Add("contains-test", checks.ContainsTest)
@@ -41,8 +45,10 @@ func DefaultRegistry() checks.Registry {
 }
 
 type certifierBuilder struct {
-	registry checks.Registry
-	checks   []string
+	checks    []string
+	config    *viper.Viper
+	overrides []string
+	registry  checks.Registry
 }
 
 func (b *certifierBuilder) SetRegistry(registry checks.Registry) CertifierBuilder {
@@ -55,6 +61,16 @@ func (b *certifierBuilder) SetChecks(checks []string) CertifierBuilder {
 	return b
 }
 
+func (b *certifierBuilder) SetConfig(config *viper.Viper) CertifierBuilder {
+	b.config = config
+	return b
+}
+
+func (b *certifierBuilder) SetOverrides(overrides []string) CertifierBuilder {
+	b.overrides = overrides
+	return b
+}
+
 func (b *certifierBuilder) Build() (Certifier, error) {
 	if len(b.checks) == 0 {
 		return nil, errors.New("no checks have been required")
@@ -64,9 +80,20 @@ func (b *certifierBuilder) Build() (Certifier, error) {
 		b.registry = defaultRegistry
 	}
 
+	if b.config == nil {
+		b.config = viper.New()
+	}
+
+	// naively override values from the configuration
+	for _, val := range b.overrides {
+		parts := strings.Split(val, "=")
+		b.config.Set(parts[0], parts[1])
+	}
+
 	return &certifier{
 		registry:       b.registry,
 		requiredChecks: b.checks,
+		config:         b.config,
 	}, nil
 }
 
diff --git a/pkg/chartverifier/checks/checks.go b/pkg/chartverifier/checks/checks.go
index 5c4c85c..16b6705 100644
--- a/pkg/chartverifier/checks/checks.go
+++ b/pkg/chartverifier/checks/checks.go
@@ -18,11 +18,14 @@ package checks
 
 import (
 	"fmt"
-	"helm.sh/helm/v3/pkg/lint"
 	"path"
+	"strconv"
 	"strings"
 
+	"helm.sh/helm/v3/pkg/lint"
+
 	"github.com/pkg/errors"
+	"github.com/spf13/viper"
 )
 
 const (
@@ -50,7 +53,13 @@ func notImplemented() (Result, error) {
 	return Result{Ok: false}, errors.New("not implemented")
 }
 
-func IsHelmV3(uri string) (Result, error) {
+func Dummy(uri string, config *viper.Viper) (Result, error) {
+	ok := config.GetBool("ok")
+	okStr := strconv.FormatBool(ok)
+	return Result{Ok: ok, Reason: "dummy.ok is set to " + okStr}, nil
+}
+
+func IsHelmV3(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -64,7 +73,7 @@ func IsHelmV3(uri string) (Result, error) {
 	return Result{Ok: isHelmV3, Reason: reason}, nil
 }
 
-func HasReadme(uri string) (Result, error) {
+func HasReadme(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -81,7 +90,7 @@ func HasReadme(uri string) (Result, error) {
 	return r, nil
 }
 
-func ContainsTest(uri string) (Result, error) {
+func ContainsTest(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -100,7 +109,7 @@ func ContainsTest(uri string) (Result, error) {
 
 }
 
-func ContainsValues(uri string) (Result, error) {
+func ContainsValues(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -116,7 +125,7 @@ func ContainsValues(uri string) (Result, error) {
 	return r, nil
 }
 
-func ContainsValuesSchema(uri string) (Result, error) {
+func ContainsValuesSchema(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -132,19 +141,19 @@ func ContainsValuesSchema(uri string) (Result, error) {
 	return r, nil
 }
 
-func KeywordsAreOpenshiftCategories(uri string) (Result, error) {
+func KeywordsAreOpenshiftCategories(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
 
-func IsCommercialChart(uri string) (Result, error) {
+func IsCommercialChart(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
 
-func IsCommunityChart(uri string) (Result, error) {
+func IsCommunityChart(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
 
-func HasMinKubeVersion(uri string) (Result, error) {
+func HasMinKubeVersion(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -160,7 +169,7 @@ func HasMinKubeVersion(uri string) (Result, error) {
 	return r, nil
 }
 
-func NotContainCRDs(uri string) (Result, error) {
+func NotContainCRDs(uri string, _ *viper.Viper) (Result, error) {
 	c, _, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -176,7 +185,7 @@ func NotContainCRDs(uri string) (Result, error) {
 	return r, nil
 }
 
-func HelmLint(uri string) (Result, error) {
+func HelmLint(uri string, _ *viper.Viper) (Result, error) {
 	c, p, err := LoadChartFromURI(uri)
 	if err != nil {
 		return Result{}, err
@@ -194,14 +203,14 @@ func HelmLint(uri string) (Result, error) {
 	return r, nil
 }
 
-func NotContainsInfraPluginsAndDrivers(uri string) (Result, error) {
+func NotContainsInfraPluginsAndDrivers(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
 
-func CanBeInstalledWithoutManualPreRequisites(uri string) (Result, error) {
+func CanBeInstalledWithoutManualPreRequisites(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
 
-func CanBeInstalledWithoutClusterAdminPrivileges(uri string) (Result, error) {
+func CanBeInstalledWithoutClusterAdminPrivileges(uri string, _ *viper.Viper) (Result, error) {
 	return notImplemented()
 }
diff --git a/pkg/chartverifier/checks/registry.go b/pkg/chartverifier/checks/registry.go
index bbc171b..bb79fce 100644
--- a/pkg/chartverifier/checks/registry.go
+++ b/pkg/chartverifier/checks/registry.go
@@ -16,6 +16,8 @@
 
 package checks
 
+import "github.com/spf13/viper"
+
 type Result struct {
 	// Ok indicates whether the result was successful or not.
 	Ok bool
@@ -24,7 +26,7 @@ type Result struct {
 	Reason string
 }
 
-type CheckFunc func(uri string) (Result, error)
+type CheckFunc func(uri string, config *viper.Viper) (Result, error)
 
 type Registry interface {
 	Get(name string) (CheckFunc, bool)
diff --git a/pkg/chartverifier/helmcertifier.go b/pkg/chartverifier/helmcertifier.go
index a512ead..49195e1 100644
--- a/pkg/chartverifier/helmcertifier.go
+++ b/pkg/chartverifier/helmcertifier.go
@@ -18,11 +18,14 @@ package chartverifier
 
 import (
 	"github.com/redhat-certification/chart-verifier/pkg/chartverifier/checks"
+	"github.com/spf13/viper"
 )
 
 type CertifierBuilder interface {
 	SetRegistry(registry checks.Registry) CertifierBuilder
 	SetChecks(checks []string) CertifierBuilder
+	SetConfig(config *viper.Viper) CertifierBuilder
+	SetOverrides([]string) CertifierBuilder
 	Build() (Certifier, error)
 }
 
diff --git a/proposals/P99999-propagate-user-defined-configuration-to-checks.md b/proposals/P99999-propagate-user-defined-configuration-to-checks.md
new file mode 100644
index 0000000..d301ea7
--- /dev/null
+++ b/proposals/P99999-propagate-user-defined-configuration-to-checks.md
@@ -0,0 +1,106 @@
+# Propagate User Defined Configuration To Checks
+
+* Proposal N.: 99999
+* Authors: isuttonl@redhat.com
+* Status: **Draft**
+
+## Abstract
+
+There are multiple use cases where propagating user defined configuration to checks is useful: to specify an OpenShift
+version to verify the chart compatibility or influence the severity the Helm linter check should consider a failure; in
+other words, any occasion the user needs to parametrize a check.
+
+## Rationale
+
+`chart-verifier` is the command (either through a Container Runtime or directly using the program directly) users
+interface to provide information regarding the verifications to be performed for a given chart, so it is expected that
+different parameters can be used at different moments in time.
+
+Those parameters can be given to the program in two ways: through the configuration file, which is already supported by
+not yet used; and through command line flags that can be used to overwrite a value defined by the configuration file.
+
+Having both mechanisms to influence the verification session is very useful for a couple of reasons:
+
+1. Configuration files can be distributed as *profiles*, for example one for OpenShift 4.6, another for OpenShift 4.7;
+1. Profile defaults can be overridden, helping developers and other power users to debug, change checks parameters
+   before committing to write a configuration/profile file.
+
+## Usage
+
+Some changes are required in the `chart-verifier` program, more specifically in the `verify` command, where the
+flag `--set` should be introduced, as in the example below:
+
+```text
+> chart-verifier verify --set compat.version=openshift-4.6 --enable compat chart.tgz
+```
+
+As the example above shows, the format of a `--set` flag is `KEY=VALUE`, where `KEY` is the path of the value in the
+configuration file and `VALUE` the value to be assigned to the configuration.
+
+In the example, `compat` is the check name, and `version` is the key the `compat` check will use to verify the
+compatibility of the given chart, in this case against the `openshift-4.6` profile.
+
+Another example related to the severity of the Helm linter check:
+
+```text
+> chart-verifier verify --set linter.failWhen=ERROR --enable linter chart.tgz
+```
+
+Both configurations could be use simultaneously by providing multiple `--set` flags:
+
+```text
+> chart-verifier verify                \
+    --set compat.version=openshift-4.6 \
+    --set linter.failWhen=ERROR        \
+    --enable compat,linter             \
+    chart.tgz
+```
+
+The configuration file could be used to store the same configuration expressed by the usage of the `--set` flag above:
+
+```yaml
+compat:
+    version: openshift-4.6
+linter:
+    failWhen: ERROR
+```
+
+The example below uses the configuration file above:
+
+```text
+> chart-verifier verify --config openshift-4.6.yaml chart.tgz
+```
+
+## Checks API
+
+Currently the `CheckFunc` type is defined as the example below:
+
+```go
+type CheckFunc func(uri string) (Result, error)
+```
+
+The current `CheckFunc` type is too simplistic, and should be modified to either receive an extra parameter containing
+the options to be considered by the check, or a more complex type composing both of this values:
+
+```go
+type CheckFunc func(uri string, config *viper.Viper) (Result, error)
+```
+
+`*viper.Viper` is used since it has already been integrated in the main program, and also because it offers the required configuration semantics to isolate checks configuration.
+
+The `opts` map is configuration for the check found in the configuration file overridden by the `--set`; in the `openshift-4.6.yaml`, the `compat` check would expect the following values in `opts`:
+
+```go
+opts := map[string]interface{}{
+    "version": "openshift-4.6"
+}
+```
+
+## Test Scenarios
+
+There are a couple of scenarios that should be tested, in order to validate the proposal implementation (assuming a `dummy` check is available):
+
+* The `dummy` check should fail by default, regardless the chart;
+* The `dummy` check should succeed when `dummy.ok` is set to `true` using the `--set` flag;
+* The `dummy` check should succeed when the configuration file configures the `dummy.ok` configuration value to `true`; and
+* The `dummy` check should fail when the configuration file configures the `dummy.ok` configuration value to `true` and `dummy.ok` is set to `false` using the `--set` flag.
\ No newline at end of file


Test patch:
diff --git a/cmd/verify_test.go b/cmd/verify_test.go
index f1c6143..2cd1526 100644
--- a/cmd/verify_test.go
+++ b/cmd/verify_test.go
@@ -21,6 +21,7 @@ import (
 	"encoding/json"
 	"testing"
 
+	"github.com/spf13/viper"
 	"github.com/stretchr/testify/require"
 	"gopkg.in/yaml.v3"
 
@@ -30,7 +31,7 @@ import (
 func TestCertify(t *testing.T) {
 
 	t.Run("Should fail when no argument is given", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -40,7 +41,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should fail when chart does not exist and argument is given", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -54,7 +55,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should fail when the chart does not exist for empty set of checks", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -67,7 +68,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should fail when the chart does not exist for single check", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -83,7 +84,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should fail when the chart exists but the single check does not", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -99,7 +100,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should succeed when the chart exists and is valid for a single check", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -123,7 +124,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should display JSON certificate when option --output and argument values are given", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -161,7 +162,7 @@ func TestCertify(t *testing.T) {
 	})
 
 	t.Run("Should display YAML certificate when option --output and argument values are given", func(t *testing.T) {
-		cmd := NewVerifyCmd()
+		cmd := NewVerifyCmd(viper.New())
 		outBuf := bytes.NewBufferString("")
 		cmd.SetOut(outBuf)
 		errBuf := bytes.NewBufferString("")
@@ -197,6 +198,164 @@ func TestCertify(t *testing.T) {
 		}
 		require.Equal(t, expected, actual)
 	})
+
+	t.Run("Should fail because 'dummy' check fails by default", func(t *testing.T) {
+		cmd := NewVerifyCmd(viper.New())
+		outBuf := bytes.NewBufferString("")
+		cmd.SetOut(outBuf)
+		errBuf := bytes.NewBufferString("")
+		cmd.SetErr(errBuf)
+
+		cmd.SetArgs([]string{
+			"-e", "dummy",
+			"-o", "yaml",
+			"../pkg/chartverifier/checks/chart-0.1.0-v3.valid.tgz",
+		})
+		require.NoError(t, cmd.Execute())
+		require.NotEmpty(t, outBuf.String())
+		outString := outBuf.String()
+		actual := map[string]interface{}{}
+		err := yaml.Unmarshal([]byte(outString), &actual)
+		require.NoError(t, err)
+
+		expected := map[string]interface{}{
+			"metadata": map[string]interface{}{
+				"chart": map[string]interface{}{
+					"name":    "chart",
+					"version": "1.16.0",
+				},
+			},
+			"ok": false,
+			"results": map[string]interface{}{
+				"dummy": map[string]interface{}{
+					"ok":     false,
+					"reason": "dummy.ok is set to false",
+				},
+			},
+		}
+		require.Equal(t, expected, actual)
+	})
+
+	t.Run("Should succeed when 'dummy' ok configuration is set to true through flag", func(t *testing.T) {
+		cmd := NewVerifyCmd(viper.New())
+		outBuf := bytes.NewBufferString("")
+		cmd.SetOut(outBuf)
+		errBuf := bytes.NewBufferString("")
+		cmd.SetErr(errBuf)
+
+		cmd.SetArgs([]string{
+			"-e", "dummy",
+			"-o", "yaml",
+			"--set", "dummy.ok=true",
+			"../pkg/chartverifier/checks/chart-0.1.0-v3.valid.tgz",
+		})
+		require.NoError(t, cmd.Execute())
+		require.NotEmpty(t, outBuf.String())
+
+		actual := map[string]interface{}{}
+		outString := outBuf.String()
+		err := yaml.Unmarshal([]byte(outString), &actual)
+		require.NoError(t, err)
+
+		expected := map[string]interface{}{
+			"metadata": map[string]interface{}{
+				"chart": map[string]interface{}{
+					"name":    "chart",
+					"version": "1.16.0",
+				},
+			},
+			"ok": true,
+			"results": map[string]interface{}{
+				"dummy": map[string]interface{}{
+					"ok":     true,
+					"reason": "dummy.ok is set to true",
+				},
+			},
+		}
+		require.Equal(t, expected, actual)
+	})
+
+	t.Run("Should succeed when 'dummy' ok configuration is set to true in global configuration", func(t *testing.T) {
+		config := viper.New()
+		config.Set("dummy.ok", true)
+
+		cmd := NewVerifyCmd(config)
+		outBuf := bytes.NewBufferString("")
+		cmd.SetOut(outBuf)
+		errBuf := bytes.NewBufferString("")
+		cmd.SetErr(errBuf)
+
+		cmd.SetArgs([]string{
+			"-e", "dummy",
+			"-o", "yaml",
+			"../pkg/chartverifier/checks/chart-0.1.0-v3.valid.tgz",
+		})
+		require.NoError(t, cmd.Execute())
+		require.NotEmpty(t, outBuf.String())
+
+		actual := map[string]interface{}{}
+		outString := outBuf.String()
+		err := yaml.Unmarshal([]byte(outString), &actual)
+		require.NoError(t, err)
+
+		expected := map[string]interface{}{
+			"metadata": map[string]interface{}{
+				"chart": map[string]interface{}{
+					"name":    "chart",
+					"version": "1.16.0",
+				},
+			},
+			"ok": true,
+			"results": map[string]interface{}{
+				"dummy": map[string]interface{}{
+					"ok":     true,
+					"reason": "dummy.ok is set to true",
+				},
+			},
+		}
+		require.Equal(t, expected, actual)
+	})
+
+	t.Run("Should fail when 'dummy' ok configuration is set to true in global configuration but overriden through flags", func(t *testing.T) {
+		config := viper.New()
+		config.Set("dummy.ok", true)
+		cmd := NewVerifyCmd(config)
+		outBuf := bytes.NewBufferString("")
+		cmd.SetOut(outBuf)
+		errBuf := bytes.NewBufferString("")
+		cmd.SetErr(errBuf)
+
+		cmd.SetArgs([]string{
+			"-e", "dummy",
+			"-o", "yaml",
+			"--set", "dummy.ok=false",
+			"../pkg/chartverifier/checks/chart-0.1.0-v3.valid.tgz",
+		})
+		require.NoError(t, cmd.Execute())
+		require.NotEmpty(t, outBuf.String())
+
+		actual := map[string]interface{}{}
+		outString := outBuf.String()
+		err := yaml.Unmarshal([]byte(outString), &actual)
+		require.NoError(t, err)
+
+		expected := map[string]interface{}{
+			"metadata": map[string]interface{}{
+				"chart": map[string]interface{}{
+					"name":    "chart",
+					"version": "1.16.0",
+				},
+			},
+			"ok": false,
+			"results": map[string]interface{}{
+				"dummy": map[string]interface{}{
+					"ok":     false,
+					"reason": "dummy.ok is set to false",
+				},
+			},
+		}
+		require.Equal(t, expected, actual)
+	})
 }
 
 func TestBuildChecks(t *testing.T) {
diff --git a/pkg/chartverifier/certifier_test.go b/pkg/chartverifier/certifier_test.go
index d036107..5b5e857 100644
--- a/pkg/chartverifier/certifier_test.go
+++ b/pkg/chartverifier/certifier_test.go
@@ -21,6 +21,7 @@ import (
 	"errors"
 	"testing"
 
+	"github.com/spf13/viper"
 	"github.com/stretchr/testify/require"
 
 	"github.com/redhat-certification/chart-verifier/pkg/chartverifier/checks"
@@ -36,15 +37,15 @@ func TestCertifier_Certify(t *testing.T) {
 
 	dummyCheckName := "dummy-check"
 
-	erroredCheck := func(uri string) (checks.Result, error) {
+	erroredCheck := func(uri string, _ *viper.Viper) (checks.Result, error) {
 		return checks.Result{}, errors.New("artificial error")
 	}
 
-	negativeCheck := func(uri string) (checks.Result, error) {
+	negativeCheck := func(uri string, _ *viper.Viper) (checks.Result, error) {
 		return checks.Result{Ok: false}, nil
 	}
 
-	positiveCheck := func(uri string) (checks.Result, error) {
+	positiveCheck := func(uri string, _ *viper.Viper) (checks.Result, error) {
 		return checks.Result{Ok: true}, nil
 	}
 
@@ -52,6 +53,7 @@ func TestCertifier_Certify(t *testing.T) {
 
 	t.Run("Should return error if check does not exist", func(t *testing.T) {
 		c := &certifier{
+			config:         viper.New(),
 			registry:       checks.NewRegistry(),
 			requiredChecks: []string{dummyCheckName},
 		}
@@ -63,6 +65,7 @@ func TestCertifier_Certify(t *testing.T) {
 
 	t.Run("Should return error if check exists and returns error", func(t *testing.T) {
 		c := &certifier{
+			config:         viper.New(),
 			registry:       checks.NewRegistry().Add(dummyCheckName, erroredCheck),
 			requiredChecks: []string{dummyCheckName},
 		}
@@ -75,6 +78,7 @@ func TestCertifier_Certify(t *testing.T) {
 	t.Run("Result should be negative if check exists and returns negative", func(t *testing.T) {
 
 		c := &certifier{
+			config:         viper.New(),
 			registry:       checks.NewRegistry().Add(dummyCheckName, negativeCheck),
 			requiredChecks: []string{dummyCheckName},
 		}
@@ -87,6 +91,7 @@ func TestCertifier_Certify(t *testing.T) {
 
 	t.Run("Result should be positive if check exists and returns positive", func(t *testing.T) {
 		c := &certifier{
+			config:         viper.New(),
 			registry:       checks.NewRegistry().Add(dummyCheckName, positiveCheck),
 			requiredChecks: []string{dummyCheckName},
 		}
diff --git a/pkg/chartverifier/checks/checks_test.go b/pkg/chartverifier/checks/checks_test.go
index d256273..51e05b5 100644
--- a/pkg/chartverifier/checks/checks_test.go
+++ b/pkg/chartverifier/checks/checks_test.go
@@ -19,6 +19,7 @@ package checks
 import (
 	"testing"
 
+	"github.com/spf13/viper"
 	"github.com/stretchr/testify/require"
 )
 
@@ -33,8 +34,9 @@ func TestIsHelmV3(t *testing.T) {
 	}
 
 	for _, tc := range positiveTestCases {
+		config := viper.New()
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := IsHelmV3(tc.uri)
+			r, err := IsHelmV3(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -47,8 +49,9 @@ func TestIsHelmV3(t *testing.T) {
 	}
 
 	for _, tc := range negativeTestCases {
+		config := viper.New()
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := IsHelmV3(tc.uri)
+			r, err := IsHelmV3(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -69,7 +72,8 @@ func TestHasReadme(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HasReadme(tc.uri)
+			config := viper.New()
+			r, err := HasReadme(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -83,7 +87,8 @@ func TestHasReadme(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HasReadme(tc.uri)
+			config := viper.New()
+			r, err := HasReadme(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -104,7 +109,8 @@ func TestContainsTest(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsTest(tc.uri)
+			config := viper.New()
+			r, err := ContainsTest(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -118,7 +124,8 @@ func TestContainsTest(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsTest(tc.uri)
+			config := viper.New()
+			r, err := ContainsTest(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -139,7 +146,8 @@ func TestHasValuesSchema(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsValuesSchema(tc.uri)
+			config := viper.New()
+			r, err := ContainsValuesSchema(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -153,7 +161,8 @@ func TestHasValuesSchema(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsValuesSchema(tc.uri)
+			config := viper.New()
+			r, err := ContainsValuesSchema(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -174,7 +183,8 @@ func TestHasValues(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsValues(tc.uri)
+			config := viper.New()
+			r, err := ContainsValues(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -188,7 +198,8 @@ func TestHasValues(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := ContainsValues(tc.uri)
+			config := viper.New()
+			r, err := ContainsValues(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -209,7 +220,8 @@ func TestHasMinKubeVersion(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HasMinKubeVersion(tc.uri)
+			config := viper.New()
+			r, err := HasMinKubeVersion(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -223,7 +235,8 @@ func TestHasMinKubeVersion(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HasMinKubeVersion(tc.uri)
+			config := viper.New()
+			r, err := HasMinKubeVersion(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -245,7 +258,8 @@ func TestNotContainCRDs(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := NotContainCRDs(tc.uri)
+			config := viper.New()
+			r, err := NotContainCRDs(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -259,7 +273,8 @@ func TestNotContainCRDs(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := NotContainCRDs(tc.uri)
+			config := viper.New()
+			r, err := NotContainCRDs(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)
@@ -280,7 +295,8 @@ func TestHelmLint(t *testing.T) {
 
 	for _, tc := range positiveTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HelmLint(tc.uri)
+			config := viper.New()
+			r, err := HelmLint(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.True(t, r.Ok)
@@ -294,7 +310,8 @@ func TestHelmLint(t *testing.T) {
 
 	for _, tc := range negativeTestCases {
 		t.Run(tc.description, func(t *testing.T) {
-			r, err := HelmLint(tc.uri)
+			config := viper.New()
+			r, err := HelmLint(tc.uri, config)
 			require.NoError(t, err)
 			require.NotNil(t, r)
 			require.False(t, r.Ok)