Package: appthreat-depscan
Introductiondep-scan is a fully open-source security audit tool based on known vulnerabilities, advisories, and license limitations for project dependencies. Both local repositories and container images are supported as the input, and the tool is ideal for CI environments with built-in build-breaker logic.FeaturesScan most application code - local repos, Linux container images, Kubernetes manifests, and OS - to identify known CVEs with prioritizationPackage vulnerability scanning is performed locally and is quite fast. No server is used!Generate Software Bill-of-Materials (SBoM) with Vulnerability Exploitability Exchange (VEX) informationPerform deep packages risk audit for dependency confusion attacks and maintenance risks (See risk audit)Vulnerability Data sourcesOSVNVDGitHubNPMLinuxvuln-list(Use--cache-os)Linux distrosAlmaLinuxDebianAlpineAmazon LinuxArch LinuxRHEL/CentOSRocky LinuxUbuntuOpenSUSE/SLESPhotonChainguardWolfi OSApplication vulnerabilities would be reported for all Linux distros and Windows. To download the full vulnerability database suitable for scanning OS, invoke dep-scan with--cache-osfor the first time. dep-scan would also download the appropriate database based on project type automatically.Usagedep-scan is ideal for use during continuous integration (CI) and as a local development tool.OCI Artifacts via ORAS cliUseORAS clito download the dep-scan binary and the vulnerability database for effortless integration. Example workflow ishere.exportVDB_HOME=depscan
mkdir-p$VDB_HOMEoraspullghcr.io/appthreat/vdb:v5-o$VDB_HOMEoraspullghcr.io/appthreat/depscan:v4-o$VDB_HOMESingle binary executablesDownload the executable binary for your operating system from thereleases page. These binary bundle the following:dep-scan with Python 3.10cdxgen with Node.js 18cdxgen binary pluginscurl-LOhttps://github.com/appthreat/depscan-bin/releases/latest/download/depscan-linux-amd64
chmod+xdepscan-linux-amd64
./depscan-linux-amd64--helpOn Windows,curl-LOhttps://github.com/appthreat/depscan-bin/releases/latest/download/depscan.exe.\depscan.exe--helpServer modedep-scan and cdxgen could be run in server mode. Use the included docker compose file to get started.gitclonehttps://github.com/AppThreat/dep-scan
dockercomposeupdepscan--server--server-host0.0.0.0--server-port7070In server mode, use/cacheendpoint to cache the vulnerability database.# This would take over 5 minutescurlhttp://0.0.0.0:7070/cacheCache all vulnerabilities including os.# This would take over 5 minutescurlhttp://0.0.0.0:7070/cache?os=trueUse the/scanendpoint to perform scans.curl--json'{"path": "/tmp/vulnerable-aws-koa-app", "type": "js"}'http://0.0.0.0:7070/scancurl--json'{"url": "https://github.com/HooliCorp/vulnerable-aws-koa-app", "type": "js"}'http://0.0.0.0:7070/scan-oapp.vex.jsonUse with ShiftLeft Scandep-scan is integrated withscan, a free and open-source SAST tool. To enable this feature simply passdepscanto the--typeargument.Referto the scan documentation for more information.-----type python,depscan,credscanThis approach should work for all CI environments supported by scan.Scanning projects locally (Python version)sudonpminstall-g@cyclonedx/cdxgen
pipinstallappthreat-depscanThis would install two commands calledcdxgenandscan.You can invoke the scan command directly with the various options.cd<projecttoscan>
depscan--src$PWD--reports-dir$PWD/reportsFull list of options are below:usage:depscan[-h][--no-banner][--cache][--cache-os][--sync][--suggest][--risk-audit][--private-nsPRIVATE_NS][-tPROJECT_TYPE][--bomBOM]-iSRC_DIR[--reports-dirREPORTS_DIR][--no-error][--deep]-h,--helpshowthishelpmessageandexit--no-bannerDonotdisplaybanner--cacheCachevulnerabilityinformationinplatformspecificuser_data_dir--cache-osCacheOSvulnerabilityinformationinplatformspecificuser_data_dir--syncSynctoreceivethelatestvulnerabilitydata.Shouldhaveinvokedcachefirst.--risk-auditPerformpackageriskaudit(slowoperation).Npmonly.--private-nsPRIVATE_NSPrivatenamespacetousewhileperformingossriskaudit.Privatepackagesshouldnotbeavailableinpublicregistriesbydefault.Commaseparatedvaluesaccepted.-tPROJECT_TYPE,--typePROJECT_TYPEOverrideprojecttypeifauto-detectionisincorrect--bomBOMExamineusingthegivenSoftwareBill-of-Materials(SBoM)fileinCycloneDXformat.Usecdxgencommandtoproduceone.-iSRC_DIR,--srcSRC_DIRSourcedirectory--reports-dirREPORTS_DIRReportsdirectory--no-errorContinueonerrortopreventbuildfrombreaking--deepPerformdeepscanbypassingthis--deepargumenttocdxgen.UsefulwhilescanningdockerimagesandOSpackages.--no-universalDepscanwouldattempttoperformasingleuniversalscaninsteadofindividualscansperlanguagetype.--no-vuln-tableDonotprintthetablewiththefulllistofvulnerabilities.Thiscanhelpreduceconsoleoutput.--threatdb-serverTHREATDB_SERVERThreatDBserverurl.Eg:https://api.sbom.cx--threatdb-usernameTHREATDB_USERNAMEThreatDBusername--threatdb-passwordTHREATDB_PASSWORDThreatDBpassword--threatdb-tokenTHREATDB_TOKENThreatDBtokenfortokenbasedsubmissionScanning containers locally (Python version)Scanlatesttag of the containershiftleft/scan-slimdepscan--no-error--cache--srcshiftleft/scan-slim-ocontainertests/depscan-scan.json-tdockerIncludelicenseto the type to perform license audit.depscan--no-error--cache--srcshiftleft/scan-slim-ocontainertests/depscan-scan.json-tdocker,licenseYou can also specify the image using the sha256 digestdepscan--no-error--srcredmine@sha256:a5c5f8a64a0d9a436a0a6941bc3fb156be0c89996add834fe33b66ebeed2439e-ocontainertests/depscan-redmine.json-tdockerYou can also save container images using docker or podman save command and pass the archive to depscan for scanning.dockersave-o/tmp/scanslim.tarshiftleft/scan-slim:latest# podman save --format oci-archive -o /tmp/scanslim.tar shiftleft/scan-slim:latestdepscan--no-error--src/tmp/scanslim.tar-oreports/depscan-scan.json-tdockerRefer to the docker tests under GitHub action workflow for this repo for more examples.Scanning projects locally (Docker container)ghcr.io/appthreat/dep-scanorpublic.ecr.aws/appthreat/dep-scan:latestcontainer image can be used to perform the scan.To scan with default settingsdockerrun--rm-v$PWD:/appghcr.io/appthreat/dep-scan--src/app--reports-dir/app/reportsUsing AWS public ECR imagedockerrun--rm-v$PWD:/apppublic.ecr.aws/appthreat/dep-scan--src/app--reports-dir/app/reportsTo scan with custom environment variables based configurationdockerrun--rm\-eVDB_HOME=/db\-eNVD_START_YEAR=2010\-eGITHUB_PAGE_COUNT=5\-eGITHUB_TOKEN=<token>\-v/tmp:/db\-v$PWD:/appghcr.io/appthreat/dep-scan--src/app--reports-dir/app/reportsIn the above example,/tmpis mounted as/dbinto the container. This directory is then specified asVDB_HOMEfor caching the vulnerability information. This way the database can be cached and reused to improve performance.Supported languages and package formatdep-scan usescdxgencommand internally to create Software Bill-of-Materials (SBoM) file for the project. This is then used for performing the scans.The following projects and package-dependency format is supported by cdxgen.LanguagePackage formatnode.jspackage-lock.json, pnpm-lock.yaml, yarn.lock, rush.js, bower.json, .min.jsjavamaven (pom.xml [1]), gradle (build.gradle, .kts), scala (sbt), bazelphpcomposer.lockpythonsetup.py, requirements.txt [2], Pipfile.lock, poetry.lock, bdist_wheel, .whl, .egg-infogobinary, go.mod, go.sum, Gopkg.lockrubyGemfile.lock, gemspecrustbinary, Cargo.toml, Cargo.lock.Net.csproj, packages.config, project.assets.json [3], packages.lock.json, .nupkgdartpubspec.lock, pubspec.yamlhaskellcabal.project.freezeelixirmix.lockc/c++conan.lock, conanfile.txtclojureClojure CLI (deps.edn), Leiningen (project.clj)docker / oci imageAll supported languages and Linux OS packagesGitHub Actions Workflows.github/workflows/*.ymlJenkins Plugins.hpi filesYAML manifestsdocker-compose, kubernetes, kustomization, skaffold, tekton etcNOTEThe docker image for dep-scan currently doesn't bundle suitable java and maven commands required for bom generation. To workaround this limitation, you can -Use python-based execution from a VM containing the correct versions for java, maven and gradle.Generate the bom file by invokingcdxgencommand locally and subsequently passing this todep-scanvia the--bomargument.Integration with CI environmentsIntegration with Azure DevOpsRefer tothis example yamlconfiguration for integrating dep-scan with Azure Pipelines. The build step would perform the scan and display the report inline as shown below:Integration with GitHub ActionsThis tool can be used with GitHub Actions using thisaction.This repo self-tests itself with both sast-scan and dep-scan! Check the GitHubworkflow fileof this repo.-name:Self dep-scanuses:AppThreat/dep-scan-action@masterenv:VDB_HOME:${{ github.workspace }}/dbGITHUB_TOKEN:${{ secrets.GITHUB_TOKEN }}Customization through environment variablesThe following environment variables can be used to customise the behaviour.VDB_HOME - Directory to use for caching database. For docker based execution, this directory should get mounted as a volume from the hostNVD_START_YEAR - Default: 2018. Supports upto 2002GITHUB_PAGE_COUNT - Default: 2. Supports upto 20GitHub Security AdvisoryTo download security advisories from GitHub, a personal access token with the following scope is necessary.read:packagesexportGITHUB_TOKEN="<PAT token>"Suggest modeFix version for each vulnerability is retrieved from the sources. Sometimes, there might be known vulnerabilities in the fix version reported. Eg: in the below screenshot the fix versions suggested for jackson-databind might contain known vulnerabilities.By passing an argument--suggestit is possible to force depscan to recheck the fix suggestions. This way the suggestion becomes more optimal for a given package group.Notice, how the new suggested version is2.9.10.5which is an optimal fix version. Please note that the optimal fix version may not be the appropriate version for your application based on compatibility.Package Risk audit--risk-auditargument enables package risk audit. Currently, only npm and pypi packages are supported in this mode. A number of risk factors are identified and assigned weights to compute a final risk score. Packages that then exceed a maximum risk score (config.pkg_max_risk_score) are presented in a table.Use--private-nsto specify the private package namespace that should be checked for dependency confusion type issues where a private package is available on public npm/pypi registry.For example, to check if private packages with namespaces @appthreat and @shiftleft are not accidentally made public, use the below argument.--private-ns appthreat,shiftleftRisk categoryDefault WeightReasonpkg_private_on_public_registry4Private package is available on a public registrypkg_min_versions2Packages with less than 3 versions represent an extreme where they could be either super stable or quite recent. Special heuristics are applied to ignore older stable packagesmod_create_min_seconds1Less than 12 hours difference between modified and creation time. This indicates that the upload had a defect that had to be rectified immediately. Sometimes, such a rapid update could also be maliciouslatest_now_min_seconds0.5Less than 12 hours difference between the latest version and the current time. Depending on the package such a latest version may or may not be desirablelatest_now_max_seconds0.5Package versions that are over 6 years old are in use. Such packages might have vulnerable dependencies that are known or yet to be foundpkg_min_maintainers2Package has less than 2 maintainers. Many opensource projects have only 1 or 2 maintainers so special heuristics are used to ignore older stable packagespkg_min_users0.25Package has less than 2 npm userspkg_install_scripts2Package runs a custom pre or post installation scripts. This is often malicious and a downside of npm.pkg_node_version0.5Package supports outdated version of node such as 0.8, 0.10, 4 or 6.x. Such projects might have prototype pollution or closure related vulnerabilitiespkg_scope4 or 0.5Packages that are used directly in the application (required scope) gets a score with a weight of 4. Optional packages get a score of 0.25deprecated1Latest version is deprecatedRefer topkg_query.py::get_category_scoremethod for the risk formula.Automatic adjustmentA parameter calledcreated_now_quarantine_secondsis used to identify packages that are safely past the quarantine period (1 year). Certain risks such aspkg_min_versionsandpkg_min_maintainersare suppressed for packages past the quarantine period. This adjustment helps reduce noise since it is unlikely that a malicious package can exist in a registry unnoticed for over a year.Configuring weightsAll parameters can be customized by using environment variables. For eg:export PKG_MIN_VERSIONS=4 to increase and set the minimum versions category to 4.Live OS scanBy passing-t os, depscan can generate an SBoM for a live operating system or a VM with OS packages and kernel information. Optionally, pass the argument--deepto generate an SBoM with both OS and application packages and to check for application vulnerabilities.All OS packages.depscan-tos-i.-oreports/depscan.jsonAll OS and application packages.depscan-tos--deep-i.-oreports/depscan.jsonLicense scandep-scan can scan the dependencies for any license limitations and report them directly on the console log. To enable license scanning set the environment variableFETCH_LICENSEtotrue.export FETCH_LICENSE=trueThe license data is sourced from choosealicense.com and is quite limited. If the license of a given package cannot be reliably matched against this list it will get silently ignored to reduce any noise. This behavior could change in the future once the detection logic gets improved.Kubernetes and Cloud appsdep-scan could auto-detect most cloud applications and Kubernetes manifest files. Pass the argument-t yaml-manifestto manually specify the type.AlternativesDependency Checkis considered the industry standard for open-source dependency scanning. After personally using this great product for many years, I decided to write my own from scratch, partly as a dedication to this project. By using a streaming database based on msgpack and using JSON schema, dep-scan is more performant than dependency check in CI environments. Plus, with support for GitHub advisory source and grafeas report export and submission, dep-scan is on track to become a next-generation dependency audit tool.Several other tools piggyback on SonatypeossindexAPI server. For some reason, I always felt uncomfortable letting a commercial company track the usage of various projects worldwide. dep-scan is, therefore, 100% private and guarantees never to perform any tracking!Trivy, grype etcThese mature projects are considered the benchmark for container, filesystem and registry scanning. dep-scan is yet to be benchmarked against these projects for results and performance. dep-scan even uses Trivy for container SBoM generation via cdxgen-binary-plugins, although this would change in the future. I aspire to make dep-scan outperform these projects with simplicity andactionable results. The CVE insights offered by dep-scan could be used for prioritization to reduce the number of package updates and container rebuilds.Discord supportThe developers could be reached via thediscordchannel.