Package: greenbone-feed-sync
greenbone-feed-syncNew script for downloading the Greenbone Community FeedInstallationRequirementsInstall using pipxInstall using pipUsageUsage on Kali LinuxCommand CompletionSettingsverbosequietconfigprivate-directorycompression-leveltypefeed-urldestination-prefixgvmd-data-destinationgvmd-data-urlnotus-destinationnotus-urlnasl-destinationnasl-urlscap-data-destinationscap-data-urlcert-data-destinationcert-data-urlreport-formats-destinationreport-formats-urlscan-configs-destinationscan-configs-urlport-lists-destinationport-lists-urlgvmd-lock-fileopenvas-lock-filefail-fastno-waitwait-intervalrsync-timeoutgroupusergreenbone-enterprise-feed-keyConfigDevelopmentMaintainerContributingLicenseInstallationRequirementsPython 3.9 and later is supported.greenbone-feed-syncrequires thersynctool being installed and available
within thePATH.On Debian based Distributions like Ubuntu and Kalirsynccan be installed viasudo apt install rsyncInstall using pipxYou can install the latest release ofgreenbone-feed-syncfrom the Python
Package Index (pypi) usingpipxpython3 -m pipx install greenbone-feed-syncOn Debian based Distributions like Ubuntu and Kalipipxitself can be
installed viasudo apt install pipxInstall using pipNOTE:Thepip installcommand does no longer work out-of-the-box in newer
distributions like Ubuntu 23.04 or Debian 12 because ofPEP 668.
Please use theinstallation via pipxinstead.You can install the latest release ofgreenbone-feed-syncfrom the
Python Package Index (pypi) usingpippython3 -m pip install greenbone-feed-syncUsageMost of the time you should just run the script without any arguments to
download the new data for all necessary feed typesNOTE:See details aboutusage on Kali Linuxsudo greenbone-feed-syncTo get verbose progress output during the data download you might increase the
verbositysudo greenbone-feed-sync -vvvIf the script is run in a cron job the output can be turned off viasudo greenbone-feed-sync --quietTo download only a specific feed content the--typeargument can be usedsudo greenbone-feed-sync --type nvtRun--helpto get information about all possible types and additional argument
optionsgreenbone-feed-sync --helpUsage on Kali LinuxWhen runninggreenbone-feed-syncas root user, for example via sudo, the
actual user and group of the process are changed to thegvmuser and group viaseteuid. This is done to
ensure thatgvmdandopenvas-scannercan read the
downloaded file contents.When using the Greenbone Community Edition installed via packages on Kali Linux
a different user and group are used. They are both named_gvminstead.
Therefore thegroupandusersettings need to be adjusted.
This can be done by using aconfig file.sudo mkdir /etc/gvm
sudo chmod +r /etc/gvm
cat <<EOF | sudo tee /etc/gvm/greenbone-feed-sync.toml
[greenbone-feed-sync]
user="_gvm"
group="_gvm"
EOF
sudo chmod +r /etc/gvm/greenbone-feed-sync.tomlCommand Completiongreenbone-feed-synccomes with support for command line completion in bash
and zsh.Setup for bash:echo"source ~/.greenbone-feed-sync-complete.bash">>~/.bashrc
greenbone-feed-sync--print-completionbash>~/.greenbone-feed-sync-complete.bashAlternatively, you can use the result of the completion command directly with
the eval function of your bash shell:eval"$(greenbone-feed-sync--print-completionbash)"Setup for zsh:echo'fpath=("$HOME/.zsh.d" $fpath)'>>~/.zsh
mkdir-p~/.zsh.d/
greenbone-feed-sync--print-completionzsh>~/.zsh.d/_greenbone_feed_syncSettingsThegreenbone-feed-syncscript is adjustable for all kind of purposes and very
flexible which content gets downloaded. Most likely you will never need to
adjust the settings because the defaults will suffice. Changing the settings
is only required for experts and testing purposes.verboseNameValueCLI Argument--verbose, -vConfig VariableverboseEnvironment VariableGREENBONE_FEED_SYNC_VERBOSEDefault Value2DescriptionLog verbosity.-vvvfor maximum verbosity.quietNameValueCLI Argument--quietConfig VariableEnvironment VariableDefault ValueDescriptionDisable all log output. Same as settingverboseorGREENBONE_FEED_SYNC_VERBOSEto 0configNameValueCLI Argument--config, -cConfig VariableEnvironment VariableDefault Value~/.config/greenbone-feed-sync.tomland/etc/gvm/greenbone-feed-sync.tomlDescriptionTOML config file to load settings from.private-directoryNameValueCLI Argument--private-directoryConfig Variableprivate-directoryEnvironment VariableGREENBONE_FEED_SYNC_PRIVATE_DIRECTORYDefault ValueDescription(Sub-)Directory to exclude from the sync which will never get deleted automatically.compression-levelNameValueCLI Argument--compression-levelConfig Variablecompression-levelEnvironment VariableGREENBONE_FEED_SYNC_COMPRESSION_LEVELDefault Value9Descriptionrsync compression level 0-9. (0 - no compression, 9 - high compression)typeNameValueCLI Argument--typeConfig VariableEnvironment VariableDefault ValueallDescriptionSpecifies which feed data should be downloaded. Possible values areall,nvt/nvts,gvmd-data,scap,cert,notus,nasl,report-format/report-formats,scan-config/scan-configsorport-list/port-lists.feed-urlNameValueCLI ArgumentConfig Variablefeed-urlEnvironment VariableGREENBONE_FEED_SYNC_URLDefault Valuersync://feed.community.greenbone.net/communityDescriptionURL to download the feed data from. Other URLs will be relative to this URL by default. For example usingrsync://example.comas feed url the notus url will bersync://example.com/vulnerability-feed/22.04/vt-data/notus/.destination-prefixNameValueCLI ArgumentConfig Variabledestination-prefixEnvironment VariableGREENBONE_FEED_SYNC_DESTINATION_PREFIXDefault Value/var/lib/DescriptionDirectory prefix to use for default feed data download destinations. Other download destinations will be relative to this path by default. For example using/opt/libas destination prefix will change the default of the notus destination to/opt/lib/notus.gvmd-data-destinationNameValueCLI Argument--gvmd-data-destinationConfig Variablegvmd-data-destinationEnvironment VariableGREENBONE_FEED_SYNC_GVMD_DATA_DESTINATIONDefault Value/var/lib/gvm/data-objects/gvmd/22.04/DescriptionDestination of the downloaded gvmd data.gvmd-data-urlNameValueCLI Argument--gvmd-data-urlConfig Variablegvmd-data-urlEnvironment VariableGREENBONE_FEED_SYNC_GVMD_DATA_URLDefault Valuersync://feed.community.greenbone.net/community/data-feed/22.04/DescriptionURL to download the gvmd data from. This includes scan-configs, report-formats and port-lists.notus-destinationNameValueCLI Argument--notus-destinationConfig Variablenotus-destinationEnvironment VariableGREENBONE_FEED_SYNC_NOTUS_DESTINATIONDefault Value/var/lib/notusDescriptionDestination of the downloaded notus data.notus-urlNameValueCLI Argument--notus-urlConfig Variablenotus-urlEnvironment VariableGREENBONE_FEED_SYNC_NOTUS_URLDefault Valuersync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/notus/DescriptionURL to download the notus data from.nasl-destinationNameValueCLI Argument--nasl-destinationConfig Variablenasl-destinationEnvironment VariableGREENBONE_FEED_SYNC_NASL_DESTINATIONDefault Value/var/lib/openvas/pluginsDescriptionDestination of the downloaded nasl data.nasl-urlNameValueCLI Argument--nasl-urlConfig Variablenasl-urlEnvironment VariableGREENBONE_FEED_SYNC_NASL_URLDefault Valuersync://feed.community.greenbone.net/community/vulnerability-feed/22.04/vt-data/nasl/DescriptionURL to download the nasl data from.scap-data-destinationNameValueCLI Argument--scap-data-destinationConfig Variablescap-data-destinationEnvironment VariableGREENBONE_FEED_SYNC_SCAP_DATA_DESTINATIONDefault Value/var/lib/gvm/scap-dataDescriptionDestination of the downloaded SCAP data.scap-data-urlNameValueCLI Argument--scap-data-urlConfig Variablescap-data-urlEnvironment VariableGREENBONE_FEED_SYNC_SCAP_DATA_URLDefault Valuersync://feed.community.greenbone.net/community/vulnerability-feed/22.04/scap-dataDescriptionURL to download the SCAP data from.cert-data-destinationNameValueCLI Argument--cert-data-destinationConfig Variablecert-data-destinationEnvironment VariableGREENBONE_FEED_SYNC_CERT_DATA_DESTINATIONDefault Value/var/lib/gvm/cert-dataDescriptionDestination of the downloaded CERT data.cert-data-urlNameValueCLI Argument--cert-data-urlConfig Variablecert-data-urlEnvironment VariableGREENBONE_FEED_SYNC_CERT_DATA_URLDefault Valuersync://feed.community.greenbone.net/community/vulnerability-feed/22.04/cert-dataDescriptionURL to download the CERT data from.report-formats-destinationNameValueCLI Argument--report-formats-destinationConfig Variablereport-formats-destinationEnvironment VariableGREENBONE_FEED_SYNC_REPORT_FORMATS_DESTINATIONDefault Value/var/lib/gvm/data-objects/gvmd/22.04/report-formatsDescriptionDestination of the downloaded report format data.report-formats-urlNameValueCLI Argument--report-formats-urlConfig Variablereport-formats-urlEnvironment VariableGREENBONE_FEED_SYNC_REPORT_FORMATS_URLDefault Valuersync://feed.community.greenbone.net/community/data-feed/22.04/report-formatsDescriptionURL to download the report format data from.scan-configs-destinationNameValueCLI Argument--scan-configs-destinationConfig Variablescan-configs-destinationEnvironment VariableGREENBONE_FEED_SYNC_SCAN_CONFIGS_DESTINATIONDefault Value/var/lib/gvm/data-objects/gvmd/22.04/scan-configsDescriptionDestination of the downloaded scan config data.scan-configs-urlNameValueCLI Argument--scan-configs-urlConfig Variablescan-configs-urlEnvironment VariableGREENBONE_FEED_SYNC_SCAN_CONFIGS_URLDefault Valuersync://feed.community.greenbone.net/community/data-feed/22.04/scan-configsDescriptionURL to download the scan config data from.port-lists-destinationNameValueCLI Argument--port-lists-destinationConfig Variableport-lists-destinationEnvironment VariableGREENBONE_FEED_SYNC_PORT_LISTS_DESTINATIONDefault Value/var/lib/gvm/data-objects/gvmd/22.04/port-listsDescriptionDestination of the downloaded port list data.port-lists-urlNameValueCLI Argument--port-lists-urlConfig Variableport-lists-urlEnvironment VariableGREENBONE_FEED_SYNC_PORT_LISTS_URLDefault Valuersync://feed.community.greenbone.net/community/data-feed/22.04/port-listsDescriptionURL to download the port list data from.gvmd-lock-fileNameValueCLI Argument--gvmd-lock-fileConfig Variablegvmd-lock-fileEnvironment VariableGREENBONE_FEED_SYNC_GVMD_LOCK_FILEDefault Value/var/lib/gvm/feed-update.lockDescriptionFile to use for locking the feed synchronization for data loaded by the gvmd daemon. Used to avoid that more then one process accesses the feed data at the same time.openvas-lock-fileNameValueCLI Argument--openvas-lock-fileConfig Variableopenvas-lock-fileEnvironment VariableGREENBONE_FEED_SYNC_OPENVAS_LOCK_FILEDefault Value/var/lib/openvas/feed-update.lockDescriptionFile to use for locking the feed synchronization for data loaded by the openvas scanner. Used to avoid that more then one process accesses the feed data at the same time.fail-fastNameValueCLI Argument--fail-fast, --failfastConfig Variablefail-fastEnvironment VariableGREENBONE_FEED_SYNC_FAIL_FASTDefault ValuefalseDescriptionStop after a first error has occurred. Otherwise the script tries to download additional data if specified.no-waitNameValueCLI Argument--no-waitConfig Variableno-waitEnvironment VariableGREENBONE_FEED_SYNC_NO_WAITDefault ValuefalseDescriptionFail directly if the lock file can't be acquired.wait-intervalNameValueCLI Argument--wait-intervalConfig Variablewait-intervalEnvironment VariableGREENBONE_FEED_SYNC_LOCK_WAIT_INTERVALDefault Value5DescriptionTime to wait in seconds after failed lock attempt before re-trying to lock the file.rsync-timeoutNameValueCLI Argument--rsync-timeoutConfig Variablersync-timeoutEnvironment VariableGREENBONE_FEED_SYNC_RSYNC_TIMEOUTDefault ValueDescriptionMaximum I/O timeout in seconds used for rsync. If no data is transferred for the specified time then rsync will exit. By default no timeout is set and the rsync default will be used.groupNameValueCLI Argument--groupConfig VariablegroupEnvironment VariableGREENBONE_FEED_SYNC_GROUPDefault ValuegvmDescriptionIf the greenbone-feed-sync script is run as root, the effective group is changed to this group name or ID.userNameValueCLI Argument--userConfig VariableuserEnvironment VariableGREENBONE_FEED_SYNC_USERDefault ValuegvmDescriptionIf the greenbone-feed-sync script is run as root, the effective user is changed to this user name or ID.greenbone-enterprise-feed-keyNameValueCLI Argument--greenbone-enterprise-feed-keyConfig Variablegreenbone-enterprise-feed-keyEnvironment VariableGREENBONE_FEED_SYNC_ENTERPRISE_FEED_KEYDefault Value/etc/gvm/greenbone-enterprise-feed-keyDescriptionFile to read the Greenbone Enterprise Feed key from. The key gives access to additional vulnerability tests for enterprise software among other advantages. SeeGreenbone Enterprise Feed and Greenbone Community Feed in Comparisonfor more details. The default URLs are adjusted according to the data in the key. If the key file does not exist it is ignored.ConfigIt is possible to use a config file for loading the settings of thegreenbone-feed-syncscript. The config file uses theTOMLformat. Without
explicitly passing a config file,greenbone-feed-synctries to load~/.config/greenbone-feed-sync.tomland if that file doesn't exist afterwards/etc/gvm/greenbone-feed-sync.toml.Example:[greenbone-feed-sync]destination-prefix="/opt/greenbone-feed"lock-file="/opt/greenbone-feed.lock"no-wait=trueDevelopmentgreenbone-feed-syncusespoetryfor its own dependency management and
build process.First install poetry via pipxpython3 -m pipx install poetryAfterwards runpoetry installin the checkout directory ofgreenbone-feed-sync(the directory containing
thepyproject.tomlfile) to install all dependencies including the packages
only required for development.Afterwards activate the git hooks for auto-formatting and linting viaautohooks.poetry run autohooks activateValidate the activated git hooks by runningpoetry run autohooks checkMaintainerThis project is maintained byGreenbone AGContributingYour contributions are highly appreciated. Pleasecreate a pull requeston GitHub. Bigger changes need to be discussed with the development team via theissues section at GitHubfirst.LicenseCopyright (C) 2022-2024Greenbone AGLicensed under theGNU General Public License v3.0 or later.