Repository: Shopify/kubeaudit (go)

Issue:
support for CRD's that extend common types
##### ISSUE TYPE
 - [ ] Bug Report
 - [X] Feature Idea

##### SUMMARY
I've noticed that Kubeaudit doesn't catch CRDS that are extending typical k8s resources, like apps/v1.
It looks like either with finagling the data out with kubectl and scanning you can still get to the original source of truth.


##### STEPS TO REPRODUCE
create a custom CRD based off apps
scan k8s
don't find anything wrong with deploy applications

##### EXPECTED RESULTS
It should find them

##### other stuff

if you guys want to point me in the direction of where in the code one can specify a custom CRD, that'll work for me and I'll add a flag to support it.

Pull request description:
✨ Adds ability to not filter CRDS, see ISSUE #373
##### Description
Adds new command flag to not filter out generated resources, which unfortunately also can include CRD based resources (like a CRD based on `Application`)
Fixes #373 

##### Type of change


- [x]  New feature :sparkles:
##### How Has This Been Tested?
I ran it against my cluster and compared results with the flag on and off and found  a lot more issues with the flag, which was my intent.

##### Checklist:

- [x] I have :tophat: my changes (A 🎩 specifically includes pulling down changes, setting them up, and manually testing the changed features and potential side effects to make sure nothing is broken)
- [x] I have performed a self-review of my own code
- [x] I have made corresponding changes to the documentation
- [ ] I have added tests that prove my fix is effective or that my feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] The test coverage did not decrease
- [x] I have signed the appropriate [Contributor License Agreement](https://cla.shopify.com/)

It is kind of difficult to test if these changes work or not, since they only apply to querying remote clusters and not local files. If you guys have a suggestion for me to add test cases, I'm happy to help or learn.


Patch:
diff --git a/.gitignore b/.gitignore
index 50bc891..d886a4d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,3 +9,4 @@ coverage.txt
 *.swp
 /vendor
 /.vscode
+.go-version
diff --git a/README.md b/README.md
index 274b89a..6c0e2e6 100644
--- a/README.md
+++ b/README.md
@@ -198,7 +198,7 @@ Auditors can also be run individually.
 | `hostns`       | Finds containers that have HostPID, HostIPC or HostNetwork enabled.                                            | [docs](docs/auditors/hostns.md)       |
 | `image`        | Finds containers which do not use the desired version of an image (via the tag) or use an image without a tag. | [docs](docs/auditors/image.md)        |
 | `limits`       | Finds containers which exceed the specified CPU and memory limits or do not specify any.                       | [docs](docs/auditors/limits.md)       |
-| `mounts`       | Finds containers that have sensitive host paths mounted.                                                       | [docs](docs/auditors/mounts.md)      |
+| `mounts`       | Finds containers that have sensitive host paths mounted.                                                       | [docs](docs/auditors/mounts.md)       |
 | `netpols`      | Finds namespaces that do not have a default-deny network policy.                                               | [docs](docs/auditors/netpols.md)      |
 | `nonroot`      | Finds containers running as root.                                                                              | [docs](docs/auditors/nonroot.md)      |
 | `privesc`      | Finds containers that allow privilege escalation.                                                              | [docs](docs/auditors/privesc.md)      |
@@ -208,14 +208,15 @@ Auditors can also be run individually.
 
 ### Global Flags
 
-| Short | Long          | Description                                                                                                                                            |
-| :---- | :------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
-|       | --format      | The output format to use (one of "pretty", "logrus", "json") (default is "pretty")                                                                     |
-| -c    | --kubeconfig  | Path to local Kubernetes config file. Only used in local mode (default is `$HOME/.kube/config`)                                                        |
-| -f    | --manifest    | Path to the yaml configuration to audit. Only used in manifest mode.                                                                                   |
-| -n    | --namespace   | Only audit resources in the specified namespace. Not currently supported in manifest mode.                                                             |
-| -m    | --minseverity | Set the lowest severity level to report (one of "error", "warning", "info") (default "info")                                                           |
-| -e    | --exitcode    | Exit code to use if there are results with severity of "error". Conventionally, 0 is used for success and all non-zero codes for an error. (default 2) |
+| Short | Long               | Description                                                                                                                                            |
+| :---- | :----------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- |
+|       | --format           | The output format to use (one of "pretty", "logrus", "json") (default is "pretty")                                                                     |
+| -c    | --kubeconfig       | Path to local Kubernetes config file. Only used in local mode (default is `$HOME/.kube/config`)                                                        |
+| -f    | --manifest         | Path to the yaml configuration to audit. Only used in manifest mode.                                                                                   |
+| -n    | --namespace        | Only audit resources in the specified namespace. Not currently supported in manifest mode.                                                             |
+| -g    | --includegenerated | Include generated resources in scan  (such as Pods generated by deployments). If you would like kubeaudit to produce results for generated resources (for example if you have custom resources or want to catch orphaned resources where the owner resource no longer exists) you can use this flag. |
+| -m    | --minseverity      | Set the lowest severity level to report (one of "error", "warning", "info") (default "info")                                                           |
+| -e    | --exitcode         | Exit code to use if there are results with severity of "error". Conventionally, 0 is used for success and all non-zero codes for an error. (default 2) |
 
 ## Configuration File
 
diff --git a/auditors/apparmor/fixtures/apparmor-annotation-init-container-enabled.yml b/auditors/apparmor/fixtures/apparmor-annotation-init-container-enabled.yml
new file mode 100644
index 0000000..8a05260
--- /dev/null
+++ b/auditors/apparmor/fixtures/apparmor-annotation-init-container-enabled.yml
@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod
+  namespace: apparmor-annotation-init-container-enabled
+  annotations:
+    container.apparmor.security.beta.kubernetes.io/container: localhost/someval
+    container.apparmor.security.beta.kubernetes.io/init-container: localhost/someval
+spec:
+  initContainers:
+  - name: init-container
+    image: scratch
+  containers:
+    - name: container
+      image: scratch
diff --git a/auditors/apparmor/fixtures/apparmor-annotation-init-container-missing.yml b/auditors/apparmor/fixtures/apparmor-annotation-init-container-missing.yml
new file mode 100644
index 0000000..072a68c
--- /dev/null
+++ b/auditors/apparmor/fixtures/apparmor-annotation-init-container-missing.yml
@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: pod
+  namespace: apparmor-annotation-init-container-missing
+  annotations:
+    container.apparmor.security.beta.kubernetes.io/container: localhost/someval
+spec:
+  initContainers:
+  - name: init-container
+    image: scratch
+  containers:
+    - name: container
+      image: scratch
diff --git a/cmd/commands/root.go b/cmd/commands/root.go
index ec133d4..6054809 100644
--- a/cmd/commands/root.go
+++ b/cmd/commands/root.go
@@ -23,6 +23,7 @@ type rootFlags struct {
 	namespace   string
 	minSeverity string
 	exitCode    int
+	includeGenerated bool
 }
 
 // RootCmd defines the shell command usage for kubeaudit.
@@ -50,6 +51,7 @@ func init() {
 	RootCmd.PersistentFlags().StringVarP(&rootConfig.minSeverity, "minseverity", "m", "info", "Set the lowest severity level to report (one of \"error\", \"warning\", \"info\")")
 	RootCmd.PersistentFlags().StringVarP(&rootConfig.format, "format", "p", "pretty", "The output format to use (one of \"pretty\", \"logrus\", \"json\")")
 	RootCmd.PersistentFlags().StringVarP(&rootConfig.namespace, "namespace", "n", apiv1.NamespaceAll, "Only audit resources in the specified namespace. Not currently supported in manifest mode.")
+	RootCmd.PersistentFlags().BoolVarP(&rootConfig.includeGenerated, "includegenerated", "g", false, "Include generated resources in scan  (eg. pods generated by deployments).")
 	RootCmd.PersistentFlags().StringVarP(&rootConfig.manifest, "manifest", "f", "", "Path to the yaml configuration to audit. Only used in manifest mode.")
 	RootCmd.PersistentFlags().IntVarP(&rootConfig.exitCode, "exitcode", "e", 2, "Exit code to use if there are results with severity of \"error\". Conventionally, 0 is used for success and all non-zero codes for an error.")
 }
@@ -101,14 +103,14 @@ func getReport(auditors ...kubeaudit.Auditable) *kubeaudit.Report {
 	}
 
 	if k8sinternal.IsRunningInCluster(k8sinternal.DefaultClient) && rootConfig.kubeConfig == "" {
-		report, err := auditor.AuditCluster(k8sinternal.ClientOptions{Namespace: rootConfig.namespace})
+		report, err := auditor.AuditCluster(k8sinternal.ClientOptions{Namespace: rootConfig.namespace, IncludeGenerated: rootConfig.includeGenerated})
 		if err != nil {
 			log.WithError(err).Fatal("Error auditing cluster")
 		}
 		return report
 	}
 
-	report, err := auditor.AuditLocal(rootConfig.kubeConfig, kubeaudit.AuditOptions{Namespace: rootConfig.namespace})
+	report, err := auditor.AuditLocal(rootConfig.kubeConfig, kubeaudit.AuditOptions{Namespace: rootConfig.namespace, IncludeGenerated: rootConfig.includeGenerated})
 	if err != nil {
 		log.WithError(err).Fatal("Error auditing cluster in local mode")
 	}
diff --git a/internal/k8sinternal/client.go b/internal/k8sinternal/client.go
index e183f4e..722468d 100644
--- a/internal/k8sinternal/client.go
+++ b/internal/k8sinternal/client.go
@@ -85,6 +85,8 @@ func IsRunningInCluster(client Client) bool {
 type ClientOptions struct {
 	// Namespace filters resources by namespace. Defaults to all namespaces.
 	Namespace string
+	// IncludeGenerated is a boolean option to include generated resources.
+	IncludeGenerated bool
 }
 
 // GetAllResources gets all supported resources from the cluster
@@ -103,8 +105,9 @@ func GetAllResources(clientset kubernetes.Interface, options ClientOptions) []k8
 	resources = append(resources, GetNamespaces(clientset, options)...)
 	resources = append(resources, GetServices(clientset, options)...)
 	resources = append(resources, GetJobs(clientset, options)...)
-
-	resources = excludeGenerated(resources)
+	if options.IncludeGenerated == false {
+		resources = excludeGenerated(resources)
+	}
 
 	return resources
 }
diff --git a/pkg/k8s/helpers.go b/pkg/k8s/helpers.go
index c4329d2..1b8cf07 100644
--- a/pkg/k8s/helpers.go
+++ b/pkg/k8s/helpers.go
@@ -17,9 +17,26 @@ func GetContainers(resource Resource) []*ContainerV1 {
 		return nil
 	}
 
-	containers := make([]*ContainerV1, len(podSpec.Containers))
+	var containers []*ContainerV1
 	for i := range podSpec.Containers {
-		containers[i] = &podSpec.Containers[i]
+		containers = append(containers, &podSpec.Containers[i])
+	}
+
+	if len(podSpec.InitContainers) > 0 {
+		containers = append(containers, GetInitContainers(resource)...)
+	}
+	return containers
+}
+
+func GetInitContainers(resource Resource) []*ContainerV1 {
+	podSpec := GetPodSpec(resource)
+	if podSpec == nil {
+		return nil
+	}
+
+	containers := make([]*ContainerV1, len(podSpec.InitContainers))
+	for i := range podSpec.InitContainers {
+		containers[i] = &podSpec.InitContainers[i]
 	}
 	return containers
 }


Test patch:
diff --git a/auditors/apparmor/apparmor_test.go b/auditors/apparmor/apparmor_test.go
index dd51cfc..a59d47b 100644
--- a/auditors/apparmor/apparmor_test.go
+++ b/auditors/apparmor/apparmor_test.go
@@ -19,6 +19,8 @@ func TestAuditAppArmor(t *testing.T) {
 	}{
 		{"apparmor-enabled.yml", nil, true},
 		{"apparmor-annotation-missing.yml", []string{AppArmorAnnotationMissing}, true},
+		{"apparmor-annotation-init-container-enabled.yml", nil, true},
+		{"apparmor-annotation-init-container-missing.yml", []string{AppArmorAnnotationMissing}, true},
 		// These are invalid manifests so we should only test it in manifest mode as kubernetes will fail to apply it
 		{"apparmor-disabled.yml", []string{AppArmorDisabled}, false},
 		{"apparmor-invalid-annotation.yml", []string{AppArmorInvalidAnnotation}, false},