Package: owasp-jenkins
OWASP Jenkins in DockerWant to automate testing your web applications and REST API service layers using the latest OWASP security toolchains and the NIST National Vulnerability Database (NVD)?This repository uses Ansible to create a docker container to hold an automatically-configured Jenkins application with theOWASP Dependency Checker,NIST NVD,Python OWASP ZAP, andOpenstack Banditinstalled. All Jenkins jobs run inside this docker container and are hosted using self-signed ssl certificates.Hopefully this will make securing your applications easier by jumpstarting your testing with theOWASP Top 10 Application Security Risks - 2017.Quickly Analyze any Repository with OWASPHere’s how to scan a repository for security issues. This will download the latesthttps://hub.docker.com/r/jayjohnson/owasp-jenkinscontainer. Please note: because there are so many known vulnerabilities to test, the container inflates to a size of about4.4 GBon disk.In this example I am testing the Bandit repositoryhttps://github.com/openstack/bandit.gitand will create theowasp-report-<date>.htmlfile in the current directory before removing the container.Check there’s nothing in the directory:ls | grep htmlPick a Repository to Scanrepo=https://github.com/openstack/bandit.gitRun the OWASP Analysis and Generate the HTML Reportdocker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-report-$(date +'%Y-%m-%d-%H-%M-%S').html"This will log something like below:d9d9c4e1945b7c0822f29aaae4db48842454ed693e1cc40d041f8362cd49cb12
Cloning into '/opt/scanrepo'...
remote: Counting objects: 5975, done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 5975 (delta 5), reused 21 (delta 0), pack-reused 5949
Receiving objects: 100% (5975/5975), 1.39 MiB | 0 bytes/s, done.
Resolving deltas: 100% (4104/4104), done.
[WARNING]: log file at /opt/owasp/ansible/"/tmp/owasp-jenkins.log" is not writeable and we cannot create it, aborting


PLAY [Running OWASP Analysis] **************************************************

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [set_fact] ****************************************************************
ok: [localhost]

TASK [Checking if this is a rebuild_nvd=0] *************************************
skipping: [localhost]

TASK [Building OWASP Arguments] ************************************************
ok: [localhost]

TASK [Running OWASP Report depchecker=/opt/tools/depcheck/dependency-check-cli/target/release/bin/dependency-check.sh owasp_args= -n --enableExperimental true --out /opt/reports/owasp-report-2018-01-10-20-21-18.html --scan /opt/scanrepo -P /opt/owasp/ansible/roles/install/files/initial-pom.xml --project analyze-this-code --data /opt/nvd] ***
changed: [localhost]

TASK [Checking if the OWASP Report=/opt/reports/owasp-report-2018-01-10-20-21-18.html exists] ***
ok: [localhost]

TASK [Verifying OWASP Report=/opt/reports/owasp-report-2018-01-10-20-21-18.html was created] ***
skipping: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=12   changed=1    unreachable=0    failed=0Verify the OWASP HTML Report Existsls | grep html
owasp-report-2018-01-10-20-21-18.htmlCleanup the Docker Containerdocker stop owasp-jenkins; docker rm owasp-jenkins
owasp-jenkins
owasp-jenkinsAnalyzing Popular GitHub RepositoriesI hope this container makes it easy to find security risks in your code and in any third-party dependencies. Here’s examples for analyzing some of the most popular GitHub repositories:Scan DjangoScan Django and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/django/django.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-django-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-django-report-*.htmlScan ReactScan React and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/facebook/react.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-react-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-react-report-*.htmlScan VueScan Vue and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/vuejs/vue.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-vue-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-vue-report-*.htmlScan AngularScan Angular and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/angular/angular.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-angular-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-angular-report-*.htmlScan Ruby on RailsScan Ruby on Rails and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/rails/rails
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-ror-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-ror-report-*.htmlScan Shadowsocks WindowsScan Shadowsocks Windows and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/shadowsocks/shadowsocks-windows.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-shadowsockswindows-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-shadowsockswindows-report-*.htmlScan LaravelScan Laravel and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/laravel/laravel
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-laravel-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-laravel-report-*.htmlScan Django REST FrameworkScan Django REST Framework and generate an OWASP HTML Report for third-party vulnerabilities:repo=https://github.com/encode/django-rest-framework.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/scanrepo" -e owasp_report_file="/opt/reports/owasp-drf-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l owasp-drf-report-*.htmlWant to generate Bandit reports for some of the most popular python projects?Scan Tensorflow and generate a Bandit HTML report (this can take a few minutes depending on your host):repo=https://github.com/tensorflow/tensorflow.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-bandit-analysis.yml -e bandit_scan_dir="/opt/scanrepo" -e bandit_report_file="/opt/reports/bandit-tf-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l bandit-tf-report-*.htmlScan Flask and generate a Bandit HTML report:repo=https://github.com/pallets/flask.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-bandit-analysis.yml -e bandit_scan_dir="/opt/scanrepo" -e bandit_report_file="/opt/reports/bandit-flask-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l bandit-flask-report-*.htmlScan Ansible and generate a Bandit HTML report (this can take a few minutes depending on your host):repo=https://github.com/ansible/ansible.git
docker run --name owasp-jenkins -p 8443:8443 -v $(pwd):/opt/reports -it -d jayjohnson/owasp-jenkins:latest && docker exec -it owasp-jenkins git clone $repo /opt/scanrepo && docker exec -it owasp-jenkins ansible-playbook -i inventories/inventory_dev run-bandit-analysis.yml -e bandit_scan_dir="/opt/scanrepo" -e bandit_report_file="/opt/reports/bandit-ab-report-$(date +'%Y-%m-%d-%H-%M-%S').html" && docker stop owasp-jenkins && docker rm owasp-jenkins
ls -l bandit-ab-report-*.htmlSetting up Jenkins for Automating your Security TestingStart the ContainerIf you want to set up the Jenkins container or onboard an application with OWASP testing you can start the container with:./start.shLogin to JenkinsThe login for the Jenkins instance is:username: adminpassword: testinghttps://localhost:8443/Running the OWASP Tools ManuallyI find it easier to initially integrate my applications with the OWASP + NIST toolchains by manually running tests from inside the container without a Jenkins job to debug at the same time.SSH into the container with:docker exec -it owasp-jenkins bashor from the base repository directory:./ssh.shConfirm you’re in the ansible directory:pwd
/opt/owasp/ansibleRun OWASP Analysis and Generate an HTML ReportThis command will analyze the repository’s/opt/owasp/owasp_jenkins/log/*.pymodules using verbose Ansible terminal output. This is helpful for figuring out what Ansible is doing under the hood. By default the Ansible playbook will create the OWASP html file inside the docker container directory:/opt/reports. This directory is set up in the compose file to auto-mount to the host’s directory./reportsfrom the repository to make sharing and viewing these html reports easier.ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="/opt/owasp/owasp_jenkins/log" -e owasp_report_file="/opt/reports/owasp-report.html" -vvvvRun Bandit Analysis and Generate an HTML ReportThis will analyze the Bandit project’s own code with the bandit analyzer and generate an html report that will be stored on the host in the./reportsdirectory.ansible-playbook -i inventories/inventory_dev run-bandit-analysis.yml -e bandit_scan_dir="/opt/owasp/venv/lib/python3.5/site-packages/bandit" -e bandit_report_file="/opt/reports/bandit-report.html" -vvvvOnboarding Your Own Application with OWASPThe Ansible playbook configures theDependency Checker parametersfor making onboarding easier even behind a corporate proxy. These are the general steps I run through to get an application automatically scanned within a Jenkins job.Changing the Runtime ParametersPlease checkout what can be overridden from the ansible-playbook cli using the-e<argname>="<argvalue>"and then port them into your Jenkins build jobs.https://github.com/jay-johnson/owasp-jenkins/blob/master/ansible/roles/install/vars/jenkins-runtime-latest.ymlTuning OWASP Runtime ArgumentsThe Dependency Checker supports numerous parameters to test and audit an application. I would recommend periodically reviewing what has changed to make sure you are using the right ones for each application:https://jeremylong.github.io/DependencyCheck/dependency-check-maven/configuration.htmlThis repository was built to analyze python, but thedefault pom.xml fileis set up with most of the availablelanguage analyzersenabled (node.js, java, .NET, ruby, php) and uses the default flag:owasp_analyzers="--enableExperimentaltrue"Setting up an OWASP pom.xml fileThere are two samplepom.xmlfiles in the repo. One is for testing with mycelery-connectorsrepository and the other is the default.https://github.com/jay-johnson/owasp-jenkins/blob/master/ansible/roles/install/files/initial-pom.xmlhttps://github.com/jay-johnson/owasp-jenkins/blob/master/ansible/roles/install/files/celery-connectors-pom.xmlThere are numerous different configurable options that each application should review to ensure they are testing their code accordingly.https://jeremylong.github.io/DependencyCheck/dependency-check-maven/index.htmlOnce you have apom.xmlready for testing you can use it with therun-owasp-analysis.ymlby adding the arguments:-eowasp_pom="<pathto your application pom.xml>"Set up OWASP Jenkins JobsI prefer to set up my Jenkins jobs using theExecute shell - Commandto configure my security toolchains in my CI/CD pipelines. These are the shell snippets for how I set up my initial OWASP jobs for a new security-ready CI/CD pipeline.NIST National Vulnerability Database Update JobThis job should run every seven days to pull in the latest updates or you can just rebuild this container (just a friendly reminder, don’t forget to back up or migrate your jobs):https://jeremylong.github.io/DependencyCheck/data/index.htmlecho "Downloading NIST National Vulnerability Database file"
. /opt/owasp/venv/bin/activate
cd /opt/owasp/ansible
ansible-playbook -i inventories/inventory_dev download-nvd.yml -vvvvRun OWASP and Bandit Analysis on any new repo PR or merged-PR JobI usually assume the Jenkins job hasWORKSPACEas the directory for the source code to check. I also try to automate email delivery by making sure the auto-generated html files are under the job’s workspace to ensure the job can send an email with the files attached for review.echo "Running OWASP Analysis on Workspace=${WORKSPACE}"
. /opt/owasp/venv/bin/activate
cd /opt/owasp/ansible

# If needed, make sure to specify the path to the repository's pom.xml:
# -e owasp_pom="/opt/owasp/ansible/roles/install/files/initial-pom.xml"
# and set the project label to match it:
# -e owasp_project_label="analyze-this-code"
ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e owasp_scan_dir="${WORKSPACE}" -e owasp_report_file="${WORKSPACE}/owasp-report.html" -vvvv

echo "Running Bandit Analysis on Workspace=${WORKSPACE}"
ansible-playbook -i inventories/inventory_dev run-bandit-analysis.yml -e bandit_scan_dir="${WORKSPACE}" -e bandit_report_file="${WORKSPACE}/bandit-report.html" -vvvvUpdate NIST Downloader and Dependency Checker Tools JobThis job will update the local, cloned repositories for the NIST NVD Downloader and Dependency Checker. This is helpful if you have to maintain an internal fork of these repositories for enhancing or modifying their testing.echo "Installing NIST National Vulnerability Database and NVD Dependency Checker using Ansible and Maven"
. /opt/owasp/venv/bin/activate
cd /opt/owasp/ansible
ansible-playbook -i inventories/inventory_dev install-tools.yml -vvvvBuild the OWASP Jenkins ContainerThis will build a large docker container (derived fromjenkins/jenkins:latest) by installing the following security packages listed below. If you want to install these later after the build you can run the Ansible playbooks as needed by commenting out the install lines of the Dockerfile (https://github.com/jay-johnson/owasp-jenkins/blob/master/Dockerfile#L84-L102).Build the container using this script in the base directory of the repository:./build.shWhile you’re waiting, here’s what is installing inside the container:OWASP WebsiteNVD Data FeedsDependency CheckerOpenstack BanditPython OWASP ZAPPython OWASP ZAP Community ScriptsContainer OWASP Coverage AnalysisSo how does this container and approach help cover applications for the2017 OWASP Top 10 Application Security Risks?The container was built to help quickly secure python application code with Bandit forStatic Application Security Testing (SAST)and ZAP forDynamic Application Security Testing (DAST). There’s tradeoffs to adding more and more tools to cover each item because while the tools might help find more security risks the tradeoff is your team just added more testing overhead to support tools and keep them updated with your application builds.This is a table to visualize how the container helps tests the OWASP Top 10:OWASP Top 10 Application Security Risks - 2017DASTSASTThird PartyA1 InjectionZAPBanditDependency CheckA2 Broken AuthenticationZAPBanditDependency CheckA3 Sensitive Data ExposureZAPBanditDependency CheckA4 XML External EntitiesZAPBanditDependency CheckA5 Broken Access ControlZAPBanditDependency CheckA6 Security MisconfigurationZAPBanditDependency CheckA7 Cross Site ScriptingZAPBanditDependency CheckA8 Insecure DeserializationNoneBanditDependency CheckA9 Using Components with Known VulnerabilitiesZAPNoneDependency CheckA10 Insufficient Logging and MonitoringNoneNoneNoneForce a Rebuild of the NVD H2 files using the Dependency CheckerIf you want to manually download the latest NVD updates you can run the included Ansible playbook from inside the container. This can take a while if you’re behind a proxy so I usually have a dedicated Jenkins job that handles updating the h2 database during off hours.ansible-playbook -i inventories/inventory_dev run-owasp-analysis.yml -e rebuild_nvd=1 -e owasp_scan_dir="/opt/owasp/owasp_jenkins/log" -vvvvCleaning up Everything on the Host before a Clean RebuildPlease be careful. This command will delete all the downloaded NIST NVD data files, maven, and the Dependency Checker tool if you have host-mounted them and commented-out the ansible-playbook install steps in the Docker container.sudo rm -rf ./docker/data/nvd/* ./docker/data/nvd/.git ./docker/data/tools/nvd/* ./docker/data/tools/nvd/.git ./docker/data/tools/depcheck/* ./docker/data/tools/depcheck/.git ./docker/data/tools/*Setting up a Development EnvironmentSetup the virtual environment with the command:virtualenv -p python3 venv && source venv/bin/activate && pip install -e .Lintingpycodestyle --max-line-length=160 --exclude=venv,build,.toxLicenseApache 2.0 - Please refer to theLICENSEfor more details