Project: secureCodeBox (Provisioning / Security & Compliance)
File: persistence-storage.md

title: "Storing Findings with Persistence Hooks" description: "Ingesting identified findings to DefectDojo or Elasticsearch" sidebar_position: 5 Let's say you need to manage a cluster of potential security risks, such as a large number of websites or your company's internal network. Your needs will exceed the limitations of manual findings handling or our build-in minio instance quite soon. The solution comes with a so-called persistence provider, which allows you to not only store your findings over a long period of time (as the name suggests), but also classify, prioritize and analyze them. This tutorial discusses two persistence providers, and , which can both be combined with the secureCodeBox. *Side-note: The tutorials are scanning scanme.nmap.org, which is a public service provided by nmap. It is meant to be used for port-scanning examples, but should not be abused with too much scanning per day. If you know that your local machine is providing meaningful findings as well, consider using localhost as target for all scans. We chose to use scanme.nmap.org in the tutorial however, because of reproducibility for most users.* All setups have been tested with Ubuntu 22.04 and Kubernetes 1.24 Make sure you have the following tools available on your system before starting: or is a widespread platform for vulnerability management and security orchestration. The secureCodeBox provides a hook that allows to directly connect the SCB to a local or remote DefectDojo instance. In this tutorial, we will at first guide you through a step-by-step setup manual. Afterward, you will learn about some features of DefectDojo and how to use them in conjunction with the secureCodeBox. We will now at first set up an instance of DefectDojo in a local minikube cluster. After we have that running, we install the secureCodeBox and the . (General, more detailed instructions can be found ) :::info At the moment DefectDojo does not provide a Docker image for arm64. As a workaround you can run a local instance of DefectDojo (see ). ::: Using minikube (for kind clusters see instructions below): ```bash git clone https://github.com/DefectDojo/django-DefectDojo cd django-DefectDojo minikube start minikube addons enable ingress helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update helm dependency update ./helm/defectdojo helm upgrade --install \ defectdojo \ ./helm/defectdojo \ --set django.ingress.enabled=true \ --set django.ingress.activateTLS=false \ --set createSecret=true \ --set createRabbitMqSecret=true \ --set createRedisSecret=true \ --set createMysqlSecret=true \ --set createPostgresqlSecret=true \ --set host="defectdojo.default.minikube.local" \ --set "alternativeHosts={localhost,defectdojo.default.minikube.local:8080,defectdojo-django.default.svc}" ``` <details> <summary>Using kind:</summary> ```shell git clone https://github.com/DefectDojo/django-DefectDojo cd django-DefectDojo kind create cluster helm repo add bitnami https://charts.bitnami.com/bitnami helm repo update helm dependency update ./helm/defectdojo helm upgrade --install \ defectdojo \ ./helm/defectdojo \ --set django.ingress.enabled=true \ --set django.ingress.activateTLS=false \ --set createSecret=true \ --set createRabbitMqSecret=true \ --set createRedisSecret=true \ --set createMysqlSecret=true \ --set createPostgresqlSecret=true \ --set host="defectdojo.default.kind.local" \ --set "alternativeHosts={localhost,defectdojo.default.kind.local:8080,defectdojo-django.default.svc}" ``` </details> After a while, you should see that all pods are running: ```bash kubectl get pods ``` To access DefectDojo locally, you probably need to add the following lines to `/etc/hosts`: ```text ::1 defectdojo.default.minikube.local 127.0.0.1 defectdojo.default.minikube.local ::1 defectdojo.default.kind.local 127.0.0.1 defectdojo.default.kind.local ``` Now, retrieve the admin password: ```bash echo "DefectDojo admin password: $(kubectl \ get secret defectdojo \ --namespace=default \ --output jsonpath='{.data.DDADMINPASSWORD}' \ | base64 --decode)" ``` After that, we can port-forward the DefectDojo service and thereby verify that the installation was successful: ```bash kubectl port-forward --namespace=default service/defectdojo-django 8080:80 ``` We are finally ready to access our DefectDojo instance! In your browser, switch to or

You should see the DefectDojo login mask. Enter `admin` as username and type the password that you received in the step before. Once logged in, you need to get your API v2 Key. Click on the person symbol in the upper-right corner and choose API v2 Key. There should be a line like `Your current API key is abcdef123456789...`. Save it for later. (General setup instructions can be found ) At first, we install the operator and a scanner: ```bash helm repo add secureCodeBox https://charts.securecodebox.io kubectl create namespace securecodebox-system helm --namespace securecodebox-system upgrade --install securecodebox-operator secureCodeBox/operator kubectl create namespace scanning helm upgrade --namespace scanning --install nmap secureCodeBox/nmap ``` To install the DefectDojo hook, we need to create a secret with the API v2 Key we retrieved before: ```bash kubectl create secret generic defectdojo-credentials -n scanning \ --from-literal="username=admin" --from-literal="apikey=<APIv2KEY>" ``` Finally, we can install the DefectDojo hook via helm: ```bash helm upgrade --namespace scanning --install dd secureCodeBox/persistence-defectdojo ``` To verify that everything works, we now start an nmap scan and check that its results are uploaded to our DefectDojo instance. Create the following file: ```yaml title="scan.yaml" apiVersion: "execution.securecodebox.io/v1" kind: Scan metadata: name: "nmap-scanme.nmap.org" spec: scanType: "nmap" parameters: scanme.nmap.org ``` We then apply it via: ```bash kubectl apply -n scanning -f scan.yaml ``` If everything was set up correctly, you should see an nmap-scanme.nmap.org engagement in the DefectDojo engagements dashboard after a while. If you experience any problems, you may find advice in the section at the end. To give you an idea of what a workflow with the secureCodeBox and DefectDojo might look like, we will take a glance at DefectDojo's findings de-duplication ability. When scanning the same target multiple times, e.g. with a , we do not want to get notified about findings that are duplicates of already existing ones. DefectDojo handles this for us, let us try it out: At first, make sure to activate : In your DefectDojo instance, click on the settings symbol in the menu bar at the left. Select Systems Settings. Next, tick the checkbox next to Deduplicate findings. Let us now verify that if we run an identical scan again, the amount of active findings does not change. Delete the scan from before in kubernetes. Note that this does not affect DefectDojo. ```bash kubectl delete scan nmap-scanme.nmap.org -n scanning ``` Re-apply it via kubectl: ```bash kubectl apply -f scan.yaml -n scanning ``` After scan and DefectDojo hook have finished, check again your local DefectDojo instance. There should not be any new Active Findings. You can still find the duplicates under Findings -> View All Findings. It is possible to overwrite the that the DefectDojo hook uses to create products and engagements. This is especially handy when using the secureCodeBox or to annotate your scans with useful names and to keep track of their origin. For the Autodiscovery, there are already pre-defined values that can be found . To get you started with a simple example, we will now create an annotation in a manually executed scan. Create the following file: ```yaml title="scanAnnotation.yaml" apiVersion: "execution.securecodebox.io/v1" kind: Scan metadata: name: "nmap-scanme.nmap.org-annotation" annotations: defectdojo.securecodebox.io/product-name: "Annotation Example" spec: scanType: "nmap" parameters: scanme.nmap.org ``` Execute it via kubectl: ```bash kubectl apply -n scanning -f scanAnnotation.yaml ``` After a while you should see a new engagement in your DefectDojo instance with product name `Annotation

enables users to store and search pretty much any possible kind of data. provides a graphical user interface to easily access the data stored in ElasticSearch. Using them in combination with the secureCodeBox is simpler than using DefectDojo, as the comes with a ready-to-use ElasticSearch and Kibana instance. At first, we create a cluster and the secureCodeBox namespace and operator: ```bash minikube start kubectl create namespace securecodebox-system helm --namespace securecodebox-system upgrade --install securecodebox-operator secureCodeBox/operator ``` Next, we simply install an exemplary scanner and the persistence-elastic hook. Note that Kibana is enabled by default. ```bash kubectl create namespace elastic helm --namespace elastic upgrade --install nmap secureCodeBox/nmap helm --namespace elastic upgrade --install persistence-elastic secureCodeBox/persistence-elastic ``` That's it! Let us run an example scan (same as above for DefectDojo): ```yaml title="scan.yaml" apiVersion: "execution.securecodebox.io/v1" kind: Scan metadata: name: "nmap-scanme.nmap.org" spec: scanType: "nmap" parameters: scanme.nmap.org ``` Apply it via kubectl: ```bash kubectl apply -n elastic -f scan.yaml ``` After the scan has finished, you can view the findings in Kibana. Create a port-forward: ```bash kubectl port-forward -n elastic services/persistence-elastic-kibana 5601:5601 ``` You can now go to and access your data in the Kibana instance! Note that unlike DefectDojo, ElasticSearch does not automatically handle duplicated findings. However, you can find pre-defined dashboards for some scanners implemented in the secureCodeBox, which makes visualizing their results straightforward. You can find a list of all available dashboards under in your local Kibana instance. It is possible to use DefectDojo and ElasticSearch at the same time. . So when using multiple persistence providers, hook execution order does matter. The problem can be solved in two ways: Either use the or set DefectDojo to . <details> <summary>Troubleshooting</summary> Connecting the SCB to a persistence provider, especially DefectDojo, can sometimes be a bit tricky. The following tips might help in case that something went wrong: Waiting: It takes some time for the DefectDojo or Kibana instance to come up. You might also have to refresh several times in order to connect to localhost after the port-forward. Verbose logging: You can view verbose output for everything in your cluster, for example via . For the following steps, you have to have installed: ```shell kubectl krew install stern kubectl stern .* kubectl stern .* --namespace scanning ``` Re-Installation of DefectDojo: Note that if anything went wrong, and you have to re-install DefectDojo in the cluster, the createSecret flags in the values.yaml file of DefectDojo must not be set. You can find more details . Server error in DefectDojo: Make sure that all pods are running. If you are continuing to experience errors after a re-installation, consider using a different cluster manager (e.g., kind instead of minikube). Using a local instance of DefectDojo rather than Kubernetes: If nothing helps, you still have the option to run DefectDojo outside your cluster (instructions ). After that, you can connect the DefectDojo hook treating it like a remote instance of DefectDojo. The helm install command for the hook would look like this: ```shell helm upgrade --install persistence-defectdojo secureCodeBox/persistence-defectdojo \ --set="defectdojo.url=$YOURLOCALIP" ``` Increasing cluster resources: Running a lot of pods at the same time can be resource-intensive. If you find that your persistence providers are not reacting in appropriate time, you might want to increase the number of CPUs and memory usage for your minikube cluster: ```shell minikube stop minikube delete minikube start --memory 8192 --cpus 4 ``` </details>