Repository: fluxcd/flux
Pull request #2532 (2019-10-20 09:04:32)

Fix some steps in README.md to install flux by helm
# Summary 
Update next steps in README.md for instruction to install flux by helm.
1. Update github URL format for https
1. Add namespace to create k8s secret

# Error details
1. If we see the logs on k8s pods, next logs can be seen (USERNAME, TOKEN, and REPOSITORY are not the real values and they should be real values).
    ```
    ts=2019-10-14T05:47:42.358161033Z caller=loop.go:101 component=sync-loop err="git repo not ready: git clone --mirror: fatal: unable to access 'https://USERNAME:TOKEN@github.com:at-ishikawa/REPOSITORY.git/': URL using bad/illegal format or missing URL, full output:\n Cloning into bare repository '/tmp/flux-gitclone859872933'...\nfatal: unable to access 'https://USERNAME:TOKEN@github.com:at-ishikawa/REPOSITORY.git/': URL using bad/illegal format or missing URL\n"
    ```

1. If we do not add namespace in flux secret, we can get following events on the result of `kubectl describe pods`.
    ```
    # kubectl -n flux describe pods -l app=flux
    ## A lot of details
    Events:
      Type     Reason     Age               From                                                       Message
      ----     ------     ----              ----                                                       -------
    ## A lot of events
      Warning  Failed     1s (x9 over 94s)  kubelet, gke-cluster-preemptible-node-pool-9d547b9e-8lj1  Error: secrets "flux-git-auth" not found
    ```


Patch:
diff --git a/chart/flux/README.md b/chart/flux/README.md
index 768d1397..c7ecdebe 100755
--- a/chart/flux/README.md
+++ b/chart/flux/README.md
@@ -81,19 +81,19 @@ to securely provide the HTTPS credentials which then can be used in the
    - [GitHub](https://help.github.com/en/articles/creating-a-personal-access-token-for-the-command-line)
    - [GitLab](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#creating-a-personal-access-token)
    - [BitBucket](https://confluence.atlassian.com/bitbucketserver/personal-access-tokens-939515499.html)
-   
+
 1. Create a secret with your `GIT_AUTHUSER` (the username the token belongs
    to) and the `GIT_AUTHKEY` you created in the first step:
 
    ```sh
-   kubectl create secret generic flux-git-auth --from-literal=GIT_AUTHUSER=<username> --from-literal=GIT_AUTHKEY=<token>
+   kubectl create secret generic flux-git-auth --namespace flux --from-literal=GIT_AUTHUSER=<username> --from-literal=GIT_AUTHKEY=<token>
    ```
 
 1. Install Flux:
 
    ```sh
    helm install --name flux \
-   --set git.url='https://$(GIT_AUTHUSER):$(GIT_AUTHKEY)@github.com:fluxcd/flux-get-started.git' \
+   --set git.url='https://$(GIT_AUTHUSER):$(GIT_AUTHKEY)@github.com/fluxcd/flux-get-started.git' \
    --set env.secretName=flux-git-auth \
    --namespace flux \
    fluxcd/flux
@@ -153,7 +153,7 @@ called `flux-ssh-config` which in turn will be mounted into a volume named
      --namespace flux \
      fluxcd/flux
      ```
-     
+
 #### Connect Flux to a Weave Cloud instance
 
 ```sh
diff --git a/docs/tutorials/driving-flux.md b/docs/tutorials/driving-flux.md
index 279d6d91..a3bad3e2 100644
--- a/docs/tutorials/driving-flux.md
+++ b/docs/tutorials/driving-flux.md
@@ -95,7 +95,7 @@ interact with the deployments. (It enables you to drive all of Flux, so have a l
 at the resulting annotation changes and make the changes in Git. This is
 GitOps after all. :-)
 
-Tell fluxctl in which namespace is Flux installed:
+Tell fluxctl in which namespace is Flux installed
 
 ```sh
 export FLUX_FORWARD_NAMESPACE=flux
@@ -104,7 +104,7 @@ export FLUX_FORWARD_NAMESPACE=flux
 To enable Flux to sync your config, you need to add the deployment key
 to your fork.
 
-Get your Flux deployment key by running:
+Get your Flux deployment key by running
 
 ```sh
 fluxctl identity
@@ -114,7 +114,7 @@ Copy/paste the key and add it to
 `https://github.com/YOUR-USER-ID/flux-get-started/settings/keys/new` and
 enable write access for it.
 
-Wait for sync to happen or run:
+Wait for sync to happen or run
 
 ```sh
 fluxctl sync
@@ -124,7 +124,7 @@ fluxctl sync
 
 After syncing, Flux will find out which workloads there are, which
 images are available and what needs doing. To find out which workloads are
-managed by Flux, run:
+managed by Flux, run
 
 ```sh
 fluxctl list-workloads -a 
@@ -132,7 +132,7 @@ fluxctl list-workloads -a
 
 Notice that `podinfo` is on `v1.3.2` and in state `automated`.
 
-To check which images are avaible for podinfo run:
+To check which images are avaible for podinfo run
 
 ```sh
 fluxctl list-images -w demo:deployment/podinfo
@@ -185,7 +185,7 @@ Again, wait for the sync to happen or run
 fluxctl sync
 ```
 
-To check which image is current, run:
+To check which image is current, run
 
 ```sh
 fluxctl list-images -w demo:deployment/podinfo
@@ -201,7 +201,7 @@ Lock deployment with a message describing why:
 fluxctl lock -w demo:deployment/podinfo -m "1.4.2 does not work for us"
 ```
 
-The resulting diff should look like this:
+The resulting diff should look like this
 
 ```diff
 --- a/workloads/podinfo-dep.yaml
@@ -224,7 +224,7 @@ Rollback to `1.4.1`. Flag `--force` is needed because the workload is locked:
 fluxctl release --force --workload demo:deployment/podinfo -i stefanprodan/podinfo:1.4.1
 ```
 
-The response should be:
+The response should be
 
 ```sh
 Submitting release ...
@@ -254,10 +254,10 @@ And that's it. At the end of this tutorial, you have automated, locked and
 annotated deployments with Flux.
 
 Another tip, if you should get stuck anywhere: check what Flux is doing. You
-can do that by simply running:
+can do that by simply running
 
 ```sh
-kubectl logs -n flux deploy/flux -f
+kubectl logs -n default deploy/flux -f
 ```
 
 If you should have any questions, find us on Slack in the [#flux
diff --git a/pkg/cluster/kubernetes/images.go b/pkg/cluster/kubernetes/images.go
index d272ba85..44db1c3c 100644
--- a/pkg/cluster/kubernetes/images.go
+++ b/pkg/cluster/kubernetes/images.go
@@ -134,19 +134,19 @@ func (c *Cluster) ImagesToFetch() registry.ImageCreds {
 	for _, ns := range namespaces {
 		seenCreds := make(map[string]registry.Credentials)
 		for kind, resourceKind := range resourceKinds {
-			workloads, err := resourceKind.getWorkloads(ctx, c, ns)
+			workloads, err := resourceKind.getWorkloads(ctx, c, ns.Name)
 			if err != nil {
 				if apierrors.IsNotFound(err) || apierrors.IsForbidden(err) {
 					// Skip unsupported or forbidden resource kinds
 					continue
 				}
-				c.logger.Log("err", errors.Wrapf(err, "getting kind %s for namespace %s", kind, ns))
+				c.logger.Log("err", errors.Wrapf(err, "getting kind %s for namespace %s", kind, ns.Name))
 			}
 
 			imageCreds := make(registry.ImageCreds)
 			for _, workload := range workloads {
-				logger := log.With(c.logger, "resource", resource.MakeID(ns, kind, workload.GetName()))
-				mergeCredentials(logger.Log, c.includeImage, c.client, ns, workload.podTemplate, imageCreds, seenCreds)
+				logger := log.With(c.logger, "resource", resource.MakeID(ns.Name, kind, workload.GetName()))
+				mergeCredentials(logger.Log, c.includeImage, c.client, ns.Name, workload.podTemplate, imageCreds, seenCreds)
 			}
 
 			// Merge creds
diff --git a/pkg/cluster/kubernetes/kubernetes.go b/pkg/cluster/kubernetes/kubernetes.go
index 2f5eca9f..0d454de2 100644
--- a/pkg/cluster/kubernetes/kubernetes.go
+++ b/pkg/cluster/kubernetes/kubernetes.go
@@ -11,6 +11,7 @@ import (
 	"github.com/go-kit/kit/log"
 	"github.com/pkg/errors"
 	"gopkg.in/yaml.v2"
+	apiv1 "k8s.io/api/core/v1"
 	apierrors "k8s.io/apimachinery/pkg/api/errors"
 	meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/client-go/discovery"
@@ -175,12 +176,12 @@ func (c *Cluster) AllWorkloads(ctx context.Context, namespace string) (res []clu
 
 	var allworkloads []cluster.Workload
 	for _, ns := range namespaces {
-		if namespace != "" && ns != namespace {
+		if namespace != "" && ns.Name != namespace {
 			continue
 		}
 
 		for kind, resourceKind := range resourceKinds {
-			workloads, err := resourceKind.getWorkloads(ctx, c, ns)
+			workloads, err := resourceKind.getWorkloads(ctx, c, ns.Name)
 			if err != nil {
 				switch {
 				case apierrors.IsNotFound(err):
@@ -197,7 +198,7 @@ func (c *Cluster) AllWorkloads(ctx context.Context, namespace string) (res []clu
 
 			for _, workload := range workloads {
 				if !isAddon(workload) {
-					id := resource.MakeID(ns, kind, workload.GetName())
+					id := resource.MakeID(ns.Name, kind, workload.GetName())
 					c.muSyncErrors.RLock()
 					workload.syncError = c.syncErrors[id]
 					c.muSyncErrors.RUnlock()
@@ -237,22 +238,16 @@ func (c *Cluster) Export(ctx context.Context) ([]byte, error) {
 	defer encoder.Close()
 
 	for _, ns := range namespaces {
-		namespace, err := c.client.CoreV1().Namespaces().Get(ns, meta_v1.GetOptions{})
-		if err != nil {
-			return nil, err
-		}
-
 		// kind & apiVersion must be set, since TypeMeta is not populated
-		namespace.Kind = "Namespace"
-		namespace.APIVersion = "v1"
-
-		err = encoder.Encode(yamlThroughJSON{namespace})
+		ns.Kind = "Namespace"
+		ns.APIVersion = "v1"
+		err := encoder.Encode(yamlThroughJSON{ns})
 		if err != nil {
 			return nil, errors.Wrap(err, "marshalling namespace to YAML")
 		}
 
 		for _, resourceKind := range resourceKinds {
-			workloads, err := resourceKind.getWorkloads(ctx, c, ns)
+			workloads, err := resourceKind.getWorkloads(ctx, c, ns.Name)
 			if err != nil {
 				switch {
 				case apierrors.IsNotFound(err):
@@ -293,9 +288,9 @@ func (c *Cluster) PublicSSHKey(regenerate bool) (ssh.PublicKey, error) {
 // the Flux instance is expected to have access to and can look for resources inside of.
 // It returns a list of all namespaces unless an explicit list of allowed namespaces
 // has been set on the Cluster instance.
-func (c *Cluster) getAllowedAndExistingNamespaces(ctx context.Context) ([]string, error) {
+func (c *Cluster) getAllowedAndExistingNamespaces(ctx context.Context) ([]apiv1.Namespace, error) {
 	if len(c.allowedNamespaces) > 0 {
-		nsList := []string{}
+		nsList := []apiv1.Namespace{}
 		for _, name := range c.allowedNamespaces {
 			if err := ctx.Err(); err != nil {
 				return nil, err
@@ -304,7 +299,7 @@ func (c *Cluster) getAllowedAndExistingNamespaces(ctx context.Context) ([]string
 			switch {
 			case err == nil:
 				c.loggedAllowedNS[name] = false // reset, so if the namespace goes away we'll log it again
-				nsList = append(nsList, ns.Name)
+				nsList = append(nsList, *ns)
 			case apierrors.IsUnauthorized(err) || apierrors.IsForbidden(err) || apierrors.IsNotFound(err):
 				if !c.loggedAllowedNS[name] {
 					c.logger.Log("warning", "cannot access allowed namespace",
@@ -321,7 +316,11 @@ func (c *Cluster) getAllowedAndExistingNamespaces(ctx context.Context) ([]string
 	if err := ctx.Err(); err != nil {
 		return nil, err
 	}
-	return []string{meta_v1.NamespaceAll}, nil
+	namespaces, err := c.client.CoreV1().Namespaces().List(meta_v1.ListOptions{})
+	if err != nil {
+		return nil, err
+	}
+	return namespaces.Items, nil
 }
 
 func (c *Cluster) IsAllowedResource(id resource.ID) bool {
diff --git a/pkg/cluster/kubernetes/sync.go b/pkg/cluster/kubernetes/sync.go
index b391d540..01790f13 100644
--- a/pkg/cluster/kubernetes/sync.go
+++ b/pkg/cluster/kubernetes/sync.go
@@ -298,7 +298,7 @@ func (c *Cluster) listAllowedResources(
 	}
 	var result []unstructured.Unstructured
 	for _, ns := range namespaces {
-		data, err := c.client.dynamicClient.Resource(gvr).Namespace(ns).List(options)
+		data, err := c.client.dynamicClient.Resource(gvr).Namespace(ns.Name).List(options)
 		if err != nil {
 			return result, err
 		}
diff --git a/pkg/git/operations.go b/pkg/git/operations.go
index dd6c533c..8cfac229 100644
--- a/pkg/git/operations.go
+++ b/pkg/git/operations.go
@@ -469,7 +469,7 @@ func findErrorMessage(output io.Reader) string {
 		case strings.HasPrefix(sc.Text(), "ERROR fatal: "): // Saw this error on ubuntu systems
 			return sc.Text()
 		case strings.HasPrefix(sc.Text(), "error:"):
-			return strings.TrimPrefix(sc.Text(), "error: ")
+			return strings.Trim(sc.Text(), "error: ")
 		}
 	}
 	return ""


Test patch:
diff --git a/pkg/cluster/kubernetes/kubernetes_test.go b/pkg/cluster/kubernetes/kubernetes_test.go
index 2d08ac0b..6ebfee91 100644
--- a/pkg/cluster/kubernetes/kubernetes_test.go
+++ b/pkg/cluster/kubernetes/kubernetes_test.go
@@ -36,7 +36,7 @@ func testGetAllowedNamespaces(t *testing.T, namespace []string, expected []strin
 
 	result := []string{}
 	for _, namespace := range namespaces {
-		result = append(result, namespace)
+		result = append(result, namespace.ObjectMeta.Name)
 	}
 
 	if reflect.DeepEqual(result, expected) != true {
@@ -45,11 +45,11 @@ func testGetAllowedNamespaces(t *testing.T, namespace []string, expected []strin
 }
 
 func TestGetAllowedNamespacesDefault(t *testing.T) {
-	testGetAllowedNamespaces(t, []string{}, []string{""}) // this will be empty string which means all namespaces
+	testGetAllowedNamespaces(t, []string{}, []string{"default", "kube-system"})
 }
 
 func TestGetAllowedNamespacesNamespacesIsNil(t *testing.T) {
-	testGetAllowedNamespaces(t, nil, []string{""}) // this will be empty string which means all namespaces
+	testGetAllowedNamespaces(t, nil, []string{"default", "kube-system"})
 }
 
 func TestGetAllowedNamespacesNamespacesSet(t *testing.T) {