neulab/tldr huggingface dataset:
To help in this deprecation task, salt provides salt.utils.versions.warn_until. The idea behind this helper function is to show the deprecation warning to the user until salt reaches the provided version. Once that provided version is equaled salt.utils.versions.warn_until will raise a RuntimeError making salt stop its execution. This stoppage is unpleasant and will remind the developer that the deprecation limit has been reached and that the code can then be safely removed.
Consider the following example:
def some_function(bar=False, foo=None): if foo is not None: salt.utils.versions.warn_until( "Aluminium", "The 'foo' argument has been deprecated and its " "functionality removed, as such, its usage is no longer " "required.", )
Development begins on Aluminium, or v3003, after the v3002 tag is applied to the master branch.  Once this occurs, all uses of the warn_until function targeting Aluminium, along with the code they are warning about should be removed from the code.
Python 2 Deprecation FAQ FAQ • Python 2 Deprecation FAQ
• Why are we deprecating Python 2?
• Was this announced/decided before making the change?
• Can I contribute Python 2 code?
• What if my OS does not include Python 3 packages? Or, how do I upgrade from Salt on Python 2 to Salt on Python 3?
• On Modern Platforms
• On Other Platforms
• Build Your Own
• pop-build distributions
• Will you support a Python 3 master with an older Python 2 minion?
• Will you support Python 2 master with a new Python 3 minion?
• How does this impact Salt-SSH support?
• What is your plan for removing Python 2 code?
• Will Salt continue to package for Python 2?
Why are we deprecating Python 2? Python 2.7.18 was the final release of Python2, released in April 2020. At this point, Python 2 will no longer receive any unpaid support. The Python core developers are focusing their efforts on improving and enhancing Python 3. Additionally, many libraries that Salt depends on have also dropped Python 2 support.
To take advantage of continued support and improvements, Salt is joining the ranks of projects that are dropping legacy Python support in favor of Python 3.
Was this announced/decided before making the change? SEP 5 was approved in April, 2019.
Can I contribute Python 2 code? For several years, Salt has been a Python 2/3 codebase, requiring Python 3 support for all contributions. The only changes contributors can expect to this process is that we will now accept Python 3-only code changes.
While there is a significant subset of Python that is compatible with both Python 3 and legacy Python, changes requiring six or otherwise removing Python 3-only code will not be accepted.
What if my OS does not include Python 3 packages? Or, how do I upgrade from Salt on Python 2 to Salt on Python 3? There are several options to upgrade to Python 3.
On Modern Platforms At this point, most modern Linux distributions have Python 3 packages. Ubuntu 20.04 LTS has moved to Python 3.8 as it’s default system Python.
On Other Platforms Build Your Own If your current distribution does not have Python 3 it’s pretty simple to use Salt to build and distribute Python 3 on your own. Installing Python 3 and pip installing Salt gives you the most control over your distribution.
pop-build distributions Another approach is to use the pop-build distribution of Salt. With the Sodium release we will begin releasing packages for Salt using pop-build, in tandem with our normal build process. The pop-build distribution will contain Salt, Python, and any necessary dependencies, which will make it trivial to have a completely supported version of Salt.
Will you support a Python 3 master with an older Python 2 minion? Yes! Our policy of keeping newer masters backwards compatible with at least the most recent minion version is not changing. You should be able to run modern Salt Masters on Python 3 with slightly older minions running on legacy Python. Of course, if you want your minions to be able to take advantage of the newest features in Salt, upgrading to Python 3 will be necessary.
Will you support Python 2 master with a new Python 3 minion? In keeping with our existing policy, we make no guarantees about older masters with newer minions. It may work, but breakages are common and should be expected.
How does this impact Salt-SSH support? Salt-SSH will need Python 3 on the target minions. We will be upgrading Salt-SSH to provide instructions or recommendations on adding Python3 if it is not detected.
What is your plan for removing Python 2 code? Legacy Python code will be gradually removed from the Salt codebase.
While we could remove most of it at once, that introduces a high level of risk. Instead, beginning with the Sodium (v3001) release, Salt will simply drop support for Python 2. PRs will no longer be required to support Python 2 before merging.
Over time, as modules are changed, six and other legacy Python syntax will be removed. It may be at some point in the future it becomes necessary to remove the last vestiges of legacy Python from Salt, but currently the plan is to take a more measured approach.
Will Salt continue to package for Python 2? No. Beginning with Sodium (v3001), Salt will no longer release packages for, or support Python 2.
Installing Salt for development Clone the repository using:
git clone https://github.com/saltstack/salt
NOTE: tags
Just cloning the repository is enough to work with Salt and make contributions. However, fetching additional tags from git is required to have Salt report the correct version for itself. To do this, first add the git repository as an upstream source:
git remote add upstream https://github.com/saltstack/salt
Fetching tags is done with the git 'fetch' utility:
git fetch --tags upstream
Create a new virtualenv:
virtualenv /path/to/your/virtualenv
Avoid making your virtualenv path too long.
On Arch Linux, where Python 3 is the default installation of Python, use the virtualenv2 command instead of virtualenv.
On Gentoo you must use --system-site-packages to enable pkg and portage_config functionality
NOTE: Using system Python modules in the virtualenv
To use already-installed python modules in virtualenv (instead of having pip download and compile new ones), run virtualenv --system-site-packages Using this method eliminates the requirement to install the salt dependencies again, although it does assume that the listed modules are all installed in the system PYTHONPATH at the time of virtualenv creation.
NOTE: Python development package
Be sure to install python devel package in order to install required Python modules. In Debian/Ubuntu run sudo apt-get install -y python-dev. In RedHat based system install python-devel
Activate the virtualenv:
source /path/to/your/virtualenv/bin/activate
Install Salt (and dependencies) into the virtualenv:
pip install pyzmq PyYAML pycrypto msgpack jinja2 psutil futures tornado pip install -e ./salt   # the path to the salt git clone from above
NOTE: Installing psutil
Python header files are required to build this module, otherwise the pip install will fail. If your distribution separates binaries and headers into separate packages, make sure that you have the headers installed. In most Linux distributions which split the headers into their own package, this can be done by installing the python-dev or python-devel package.  For other platforms, the package will likely be similarly named.
NOTE: Installing dependencies on macOS.
You can install needed dependencies on macOS using homebrew or macports.  See macOS Installation
WARNING: Installing on RedHat-based Distros
If installing from pip (or from source using setup.py install), be advised that the yum-utils package is needed for Salt to manage packages on RedHat-based systems.
Running a self-contained development version During development it is easiest to be able to run the Salt master and minion that are installed in the virtualenv you created above, and also to have all the configuration, log, and cache files contained in the virtualenv as well.
The /path/to/your/virtualenv referenced multiple times below is also available in the variable $VIRTUAL_ENV once the virtual environment is activated.
Copy the master and minion config files into your virtualenv:
mkdir -p /path/to/your/virtualenv/etc/salt/pki/{master,minion} cp ./salt/conf/master ./salt/conf/minion /path/to/your/virtualenv/etc/salt/
Edit the master config file:
1. Uncomment and change the user: root value to your own user.
2. Uncomment and change the root_dir: / value to point to /path/to/your/virtualenv.
3. Uncomment and change the pki_dir: /etc/salt/pki/master value to point to /path/to/your/virtualenv/etc/salt/pki/master
4. If you are running version 0.11.1 or older, uncomment, and change the pidfile: /var/run/salt-master.pid value to point to /path/to/your/virtualenv/salt-master.pid.
5. If you are also running a non-development version of Salt you will have to change the publish_port and ret_port values as well.
Edit the minion config file:
1. Repeat the edits you made in the master config for the user and root_dir values as well as any port changes.
2. Uncomment and change the pki_dir: /etc/salt/pki/minion value to point to /path/to/your/virtualenv/etc/salt/pki/minion
3. If you are running version 0.11.1 or older, uncomment, and change the pidfile: /var/run/salt-minion.pid value to point to /path/to/your/virtualenv/salt-minion.pid.
4. Uncomment and change the master: salt value to point at localhost.
5. Uncomment and change the id: value to something descriptive like "saltdev". This isn't strictly necessary but it will serve as a reminder of which Salt installation you are working with.
6. If you changed the ret_port value in the master config because you are also running a non-development version of Salt, then you will have to change the master_port value in the minion config to match.
NOTE: Using salt-call with a Standalone Minion
If you plan to run salt-call with this self-contained development environment in a masterless setup, you should invoke salt-call with -c /path/to/your/virtualenv/etc/salt so that salt can find the minion config file. Without the -c option, Salt finds its config files in /etc/salt.
Start the master and minion, accept the minion's key, and verify your local Salt installation is working:
cd /path/to/your/virtualenv salt-master -c ./etc/salt -d salt-minion -c ./etc/salt -d salt-key -c ./etc/salt -L salt-key -c ./etc/salt -A salt -c ./etc/salt '*' test.version
Running the master and minion in debug mode can be helpful when developing. To do this, add -l debug to the calls to salt-master and salt-minion.  If you would like to log to the console instead of to the log file, remove the -d.
NOTE: Too long socket path?
Once the minion starts, you may see an error like the following:
zmq.core.error.ZMQError: ipc path "/path/to/your/virtualenv/ var/run/salt/minion/minion_event_7824dcbcfd7a8f6755939af70b96249f_pub.ipc" is longer than 107 characters (sizeof(sockaddr_un.sun_path)).
2. Edit the sock_dir minion config variable and reduce its length. Remember that this path is relative to the value you set in root_dir.
NOTE: File descriptor limits
Ensure that the system open file limit is raised to at least 2047:
# check your current limit ulimit -n
# raise the limit. persists only until reboot # use 'limit descriptors 2047' for c-shell ulimit -n 2047
To set file descriptors on macOS, refer to the macOS Installation instructions.
Changing Default Paths Instead of updating your configuration files to point to the new root directory and having to pass the new configuration directory path to all of Salt's CLI tools, you can explicitly tweak the default system paths that Salt expects:
GENERATE_SALT_SYSPATHS=1 pip install --global-option='--salt-root-dir=/path/to/your/virtualenv/' \ -e ./salt   # the path to the salt git clone from above
You can now call all of Salt's CLI tools without explicitly passing the configuration directory.
Additional Options If you want to distribute your virtualenv, you probably don't want to include Salt's clone .git/ directory, and, without it, Salt won't report the accurate version. You can tell setup.py to generate the hardcoded version information which is distributable:
GENERATE_SALT_SYSPATHS=1 WRITE_SALT_VERSION=1 pip install --global-option='--salt-root-dir=/path/to/your/virtualenv/' \ -e ./salt   # the path to the salt git clone from above
Instead of passing those two environmental variables, you can just pass a single one which will trigger the other two:
MIMIC_SALT_INSTALL=1 pip install --global-option='--salt-root-dir=/path/to/your/virtualenv/' \ -e ./salt   # the path to the salt git clone from above
This last one will grant you an editable salt installation with hardcoded system paths and version information.
Installing Salt from the Python Package Index If you are installing using easy_install, you will need to define a USE_SETUPTOOLS environment variable, otherwise dependencies will not be installed:
USE_SETUPTOOLS=1 easy_install salt
Editing and previewing the documentation You need sphinx-build command to build the docs. In Debian/Ubuntu this is provided in the python-sphinx package. Sphinx can also be installed to a virtualenv using pip:
pip install Sphinx==1.3.1
Change to salt documentation directory, then:
cd doc; make html
• This will build the HTML docs. Run make without any arguments to see the available make targets, which include html, man, and text.
• The docs then are built within the docs/_build/ folder. To update the docs after making changes, run make again.
• The docs use reStructuredText for markup.  See a live demo at http://rst.ninjs.org/.
• The help information on each module or state is culled from the python code that runs for that piece. Find them in salt/modules/ or salt/states/.
• To build the docs on Arch Linux, the python2-sphinx package is required. Additionally, it is necessary to tell make where to find the proper sphinx-build binary, like so:
make SPHINXBUILD=sphinx-build2 html
• To build the docs on RHEL/CentOS 6, the python-sphinx10 package must be installed from EPEL, and the following make command must be used:
make SPHINXBUILD=sphinx-build html
Once you've updated the documentation, you can run the following command to launch a simple Python HTTP server to see your changes:
cd _build/html; python -m SimpleHTTPServer
Running unit and integration tests Run the test suite with following command:
./setup.py test
See here for more information regarding the test suite.
Issue and Pull Request Labeling System SaltStack uses several labeling schemes to help facilitate code contributions and bug resolution. See the Labels and Milestones documentation for more information.
GitHub Labels and Milestones SaltStack uses several label categories, as well as milestones, to triage incoming issues and pull requests in the GitHub issue tracker.  Labels are used to sort issues by type, priority, severity, status, functional area, functional group, and targeted release and pull requests by status, functional area, functional group, type of change, and test status. Milestones are used to indicate whether an issue is fully triaged or is scheduled to be fixed by SaltStack in an upcoming sprint.
Milestones All issues are assigned to a milestone, whereas pull requests are almost never assigned to a milestone as the mean lifetime of pull requests is short enough that there is no need to track them temporally.
SaltStack uses milestones to indicate which issues are blocked on submitter or upstream actions, are approved, or are scheduled to be fixed or implemented in an upcoming sprint.  If an issue is not attached to a sprint milestone, you are welcome to work on it at your own desire and convenience.  If it is attached to a sprint milestone and you have already begun working on it or have a solution in mind or have other ideas related to the issue, you are encouraged to coordinate with the assignee via the GitHub issue tracker to create the best possible solution or implementation.
• Approved - The issue has been validated and has all necessary information.
• Blocked - The issue is waiting on actions by parties outside of SaltStack, such as receiving more information from the submitter or resolution of an upstream issue. This milestone is usually applied in conjunction with the labels Info Needed, Question, Expected Behavior, Won't Fix For Now, or Upstream Bug.
Labels Labels are used to sort and describe issues and pull requests.  Some labels are usually reserved for one or the other, though most labels may be applied to both.
New issues will receive at least one label and a milestone, and new pull requests will receive at least one label.  Except for the functional area and functional group label categories, issues will generally receive only up to one label per category.
Type Issues are categorized into one of several types.  Type labels are almost never used for pull requests.  GitHub treats pull requests like issues in many ways, so a pull request could be considered an issue with an implicit Pull Request type label applied.
• Feature - The issue is a request for new functionality including changes, enhancements, refactors, etc.
• Bug - The issue documents broken, incorrect, or confusing behavior. This label is always accompanied by a severity label.
• Duplicate - The issue is a duplicate of another feature request or bug report.
• Upstream Bug - The issue is a result of an upstream issue.
• Question - The issue is more of a question than a request for new features or a report of broken features, but can sometimes lead to further discussion or changes of confusing or incongruous behavior or documentation.
• Expected Behavior - The issue is a bug report of intended functionality.
Priority An issue's priority is relative to its functional area.  If a bug report, for example, about gitfs indicates that all users of gitfs will encounter this bug, then a P1 label will be applied, even though users who are not using gitfs will not encounter the bug.  If a feature is requested by many users, it may be given a high priority.
• P1 - The issue will be seen by all users.
• P2 - The issue will be seen by most users.
• P3 - The issue will be seen by about half of users.
• P4 - The issue will not be seen by most users.  Usually the issue is a very specific use case or corner case.
Severity Severity labels are almost always only applied to issues labeled Bug.
• Blocker - The issue is blocking an impending release.
• Critical - The issue causes data loss, crashes or hangs salt processes, makes the system unresponsive, etc.
• High Severity - The issue reports incorrect functionality, bad functionality, a confusing user experience, etc.
• Medium Severity - The issue reports cosmetic items, formatting, spelling, colors, etc.
Functional Area Many major components of Salt have corresponding GitHub labels.  These labels are applied to all issues and pull requests as is reasonably appropriate.  They are useful in organizing issues and pull requests according to the source code relevant to issues or the source code changed by pull requests.
• Execution Module
• File Servers
• Multi-Master
• Packaging  Related to packaging of Salt, not Salt's support for package management.
• RAET
• Returners
• Runners
• SPM
• Salt-API
• Salt-Cloud
• Salt-SSH
• Salt-Syndic
• State Module
• Tests
• Transport
• ZMQ
Functional Group These labels sort issues and pull requests according to the internal SaltStack engineering teams.
• Core - The issue or pull request relates to code that is central or existential to Salt itself.
• Platform - The issue or pull request relates to support and integration with various platforms like traditional operating systems as well as containers, platform-based utilities like filesystems, command schedulers, etc., and system-based applications like webservers, databases, etc.
• RIoT - The issue or pull request relates to support and integration with various abstract systems like cloud providers, hypervisors, API-based services, etc.
• Console - The issue or pull request relates to the SaltStack enterprise console.
• Documentation - The issue or pull request relates to documentation.
Status Status labels are used to define and track the state of issues and pull requests.  Not all potential statuses correspond to a label, but some statuses are common enough that labels have been created for them.  If an issue has not been moved beyond the Blocked milestone, it is very likely that it will only have a status label.
• Cannot Reproduce - The issue is a bug and has been reviewed by a SaltStack engineer, but it cannot be replicated with the provided information and context.  Those involved with the bug will need to work through additional ideas until the bug can be isolated and verified.
• Confirmed - The issue is a bug and has been confirmed by a SaltStack engineer, who often documents a minimal working example that reproduces the bug.
• Fixed Pending Verification - The issue is a bug and has been fixed by one or more pull requests, which should link to the issue.  Closure of the issue is contingent upon confirmation of resolution from the submitter.  If the submitter reports a negative confirmation, this label is removed.  If no response is given after a few weeks, then the issue will be assumed fixed and closed.
• Info Needed - The issue needs more information before it can be verified and resolved.  For a feature request this may include a description of the use cases.  Almost all bug reports need to include at least the versions of salt and its dependencies, the system type and version, commands used, debug logs, error messages, and relevant configs.
• Pending Changes - The pull request needs additional changes before it can be merged.
• Pending Discussion - The issue or pull request needs more discussion before it can be closed or merged.  The status of the issue or pull request is not clear or apparent enough for definite action to be taken, or additional input from SaltStack, the submitter, or another party has been requested.
If the issue is not a pull request, once the discussion has arrived at a cogent conclusion, this label will be removed and the issue will be accepted.  If it is a pull request, the results of the discussion may require additional changes and thus, a Pending Changes label.
• won't-fix - The issue is legitimate, but it is not something the Salt core team is currently able or willing to fix or implement.  Issues having this label may be revisited in the future, or solved by a Salt community member.
Test Status These labels relate to the status of the automated tests that run on pull requests.
• has-failing-test - The PR currently has one or more failing tests that prevent the PR from being merged.
• Needs Testcase - The PR has code changes, but lack any automated tests. These PRs need automated tests written before they may be merged.
Other These labels indicate miscellaneous issue types or statuses that are common or important enough to be tracked and sorted with labels.
• Awesome - The pull request implements an especially well crafted solution, or a very difficult but necessary change.
• Help Wanted - The issue appears to have a simple solution.  Issues having this label should be a good starting place for new contributors to Salt.
• Regression - The issue is a bug that breaks functionality known to work in previous releases.
• Story - The issue is used by a SaltStack engineer to track progress on multiple related issues in a single place.
• Stretch - The issue is an optional goal for the current sprint but may not be delivered.
• ZD - The issue is related to a Zendesk customer support ticket.
• <Release> - The issue is scheduled to be implemented by <Release>.  See here for a discussion of Salt's release codenames.
Logging Internals TODO
Package Providers This page contains guidelines for writing package providers.
Package Functions One of the most important features of Salt is package management. There is no shortage of package managers, so in the interest of providing a consistent experience in pkg states, there are certain functions that should be present in a package provider. Note that these are subject to change as new features are added or existing features are enhanced.
list_pkgs This function should declare an empty dict, and then add packages to it by calling pkg_resource.add_pkg, like so:
__salt__["pkg_resource.add_pkg"](ret, name, version)
The last thing that should be done before returning is to execute pkg_resource.sort_pkglist. This function does not presently do anything to the return dict, but will be used in future versions of Salt.
__salt__["pkg_resource.sort_pkglist"](ret)
list_pkgs returns a dictionary of installed packages, with the keys being the package names and the values being the version installed. Example return data:
{"foo": "1.2.3-4", "bar": "5.6.7-8"}
latest_version Accepts an arbitrary number of arguments. Each argument is a package name. The return value for a package will be an empty string if the package is not found or if the package is up-to-date. The only case in which a non-empty string is returned is if the package is available for new installation (i.e. not already installed) or if there is an upgrade available.
If only one argument was passed, this function return a string, otherwise a dict of name/version pairs is returned.
This function must also accept **kwargs, in order to receive the fromrepo and repo keyword arguments from pkg states. Where supported, these arguments should be used to find the install/upgrade candidate in the specified repository. The fromrepo kwarg takes precedence over repo, so if both of those kwargs are present, the repository specified in fromrepo should be used. However, if repo is used instead of fromrepo, it should still work, to preserve backwards compatibility with older versions of Salt.
version Like latest_version, accepts an arbitrary number of arguments and returns a string if a single package name was passed, or a dict of name/value pairs if more than one was passed. The only difference is that the return values are the currently-installed versions of whatever packages are passed. If the package is not installed, an empty string is returned for that package.
upgrade_available Deprecated and destined to be removed. For now, should just do the following:
def myfunc(): return __salt__["pkg.latest_version"](name) != ""
install The following arguments are required and should default to None:
1. name (for single-package pkg states)
2. pkgs (for multiple-package pkg states)
3. sources (for binary package file installation)
The first thing that this function should do is call pkg_resource.parse_targets (see below). This function will convert the SLS input into a more easily parsed data structure. pkg_resource.parse_targets may need to be modified to support your new package provider, as it does things like parsing package metadata which cannot be done for every package management system.
pkg_params, pkg_type = __salt__["pkg_resource.parse_targets"](name, pkgs, sources)
Two values will be returned to the install function. The first of them will be a dictionary. The keys of this dictionary will be package names, though the values will differ depending on what kind of installation is being done:
• If name was provided (and pkgs was not), then there will be a single key in the dictionary, and its value will be None. Once the data has been returned, if the version keyword argument was provided, then it should replace the None value in the dictionary.
• If pkgs was provided, then name is ignored, and the dictionary will contain one entry for each package in the pkgs list. The values in the dictionary will be None if a version was not specified for the package, and the desired version if specified. See the Multiple Package Installation Options section of the pkg.installed state for more info.
• If sources was provided, then name is ignored, and the dictionary values will be the path/URI for the package.
The second return value will be a string with two possible values: repository or file. The install function can use this value (if necessary) to build the proper command to install the targeted package(s).
Both before and after the installing the target(s), you should run list_pkgs to obtain a list of the installed packages. You should then return the output of salt.utils.data.compare_dicts():
def myfunc(): return salt.utils.data.compare_dicts(old, new)
remove Removes the passed package and return a list of the packages removed.
Package Repo Functions There are some functions provided by pkg which are specific to package repositories, and not to packages themselves. When writing modules for new package managers, these functions should be made available as stated below, in order to provide compatibility with the pkgrepo state.
All repo functions should accept a basedir option, which defines which directory repository configuration should be found in. The default for this is dictated by the repo manager that is being used, and rarely needs to be changed.
basedir = "/etc/yum.repos.d" __salt__["pkg.list_repos"](basedir)
list_repos Lists the repositories that are currently configured on this system.
__salt__["pkg.list_repos"]()
Returns a dictionary, in the following format:
{'reponame': 'config_key_1': 'config value 1', 'config_key_2': 'config value 2', 'config_key_3': ['list item 1 (when appropriate)', 'list item 2 (when appropriate)]}
get_repo Displays all local configuration for a specific repository.
__salt__["pkg.get_repo"](repo="myrepo")
The information is formatted in much the same way as list_repos, but is specific to only one repo.
{'config_key_1': 'config value 1', 'config_key_2': 'config value 2', 'config_key_3': ['list item 1 (when appropriate)', 'list item 2 (when appropriate)]}
del_repo Removes the local configuration for a specific repository. Requires a repo argument, which must match the locally configured name. This function returns a string, which informs the user as to whether or not the operation was a success.
__salt__["pkg.del_repo"](repo="myrepo")
mod_repo Modify the local configuration for one or more option for a configured repo.  This is also the way to create new repository configuration on the local system; if a repo is specified which does not yet exist, it will be created.
The options specified for this function are specific to the system; please refer to the documentation for your specific repo manager for specifics.
__salt__["pkg.mod_repo"](repo="myrepo", url="http://myurl.com/repo")
Low-Package Functions In general, the standard package functions as describes above will meet your needs. These functions use the system's native repo manager (for instance, yum or the apt tools). In most cases, the repo manager is actually separate from the package manager. For instance, yum is usually a front-end for rpm, and apt is usually a front-end for dpkg. When possible, the package functions that use those package managers directly should do so through the low package functions.
It is normal and sane for pkg to make calls to lowpkgs, but lowpkg must never make calls to pkg. This is affects functions which are required by both pkg and lowpkg, but the technique in pkg is more performant than what is available to lowpkg. When this is the case, the lowpkg function that requires that technique must still use the lowpkg version.
list_pkgs Returns a dict of packages installed, including the package name and version.  Can accept a list of packages; if none are specified, then all installed packages will be listed.
installed = __salt__["lowpkg.list_pkgs"]("foo", "bar")
verify Many (but not all) package management systems provide a way to verify that the files installed by the package manager have or have not changed. This function accepts a list of packages; if none are specified, all packages will be included.
installed = __salt__["lowpkg.verify"]("httpd")
{ "/etc/httpd/conf/httpd.conf": { "mismatch": ["size", "md5sum", "mtime"], "type": "config", } }
file_list Lists all of the files installed by all packages specified. If not packages are specified, then all files for all known packages are returned.
installed = __salt__["lowpkg.file_list"]("httpd", "apache")
This function does not return which files belong to which packages; all files are returned as one giant list (hence the file_list function name. However, This information is still returned inside of a dict, so that it can provide any errors to the user in a sane manner.
{ "errors": ["package apache is not installed"], "files": ["/etc/httpd", "/etc/httpd/conf", "/etc/httpd/conf.d", "...SNIP..."], }
file_dict Lists all of the files installed by all packages specified. If not packages are specified, then all files for all known packages are returned.
installed = __salt__["lowpkg.file_dict"]("httpd", "apache", "kernel")
Unlike file_list, this function will break down which files belong to which packages. It will also return errors in the same manner as file_list.
{ "errors": ["package apache is not installed"], "packages": { "httpd": ["/etc/httpd", "/etc/httpd/conf", "...SNIP..."], "kernel": [ "/boot/.vmlinuz-2.6.32-279.el6.x86_64.hmac", "/boot/System.map-2.6.32-279.el6.x86_64", "...SNIP...", ], }, }
Pull Requests Salt is a large software project with many developers working together. We encourage all Salt users to contribute new features, bug fixes and documentation fixes. For those who haven't contributed to a large software project before we encourage you to consider the following questions when preparing a pull request.
This isn't an exhaustive list and these aren't necessarily hard and fast rules, but these are things we consider when reviewing a pull request.
• Does this change work on all platforms? In cases where it does not, is an appropriate and easy-to-understand reason presented to the user? Is it documented as-such? Have we thought about all the possible ways this code might be used and accounted as best we can for them?
• Will this code work on versions of all Python we support? Will it work on future versions?
• Are Python reserved keywords used? Are variables named in a way that will make it easy for the next person to understand what's going on?
• Does this code present a security risk in any way? What is the worst possible thing that an attacker could do with this code? If dangerous cases are possible, is it appropriate to document them? If so, has this been done?  Would this change pass muster with a professional security audit? Is it obvious to a person using this code what the risks are?
• Is it readable? Does it conform to our style guide? Is the code documented such that the next person who comes along will be able to read and understand it? Most especially, are edge-cases documented to avoid regressions? Will it be immediately evident to the next person who comes along why this change was made?
• If appropriate, has the person who wrote the code which is being modified been notified and included in the process?
• What are the performance implications of this change? Is there a more efficient way to structure the logic and if so, does making the change balance itself against readability in a sensible way? Do the performance characteristics of the code change based on the way it is being invoked (i.e., through an API or various command-line tools.) Will it be easy to profile this change if it might be a problem?
• Are caveats considered and documented in the change?
• Will the code scale? More critically, will it scale in both directions? Salt runs in data-centers and on Raspberry Pi installations in the Sahara. It needs to work on big servers and tiny devices.
• Is appropriate documentation written both in public-facing docs and in-line?  How will the user know how to use this? What will they do if it doesn't work as expected? Is this something a new user will understand? Can a user know all they need to about this functionality by reading the public docs?
• Is this a change in behavior? If so, is it in the appropriate branch? Are deprecation warnings necessary? Have those changes been fully documented?  Have we fully thought through what implications a change in behavior might have?
• How has the code been tested? If appropriate are there automated tests which cover this? Is it likely to regress? If so, how has the potential of that regression been mitigated? What is the plan for ensuring that this code works going forward?
• If it's asynchronous code, what is the potential for a race condition?
• Is this code an original work? If it's borrowed from another project or found online are the appropriate licensing/attribution considerations handled?
• Is the reason for the change fully explained in the PR? If not for review, this is necessary so that somebody in the future can go back and figure out why it was necessary.
• Is the intended behavior of the change clear? How will that behavior be known to future contributors and to users?
• Does this code handle errors in a reasonable way? Have we gone back through the stack as much as possible to make sure that an error cannot be raised that we do not account for? Are errors tested for as well as proper functionality?
• If the code relies on external libraries, do we properly handle old versions of them? Do we require a specific version and if so is this version check implemented? Is the library available on the same platforms that module in question claims to support? If the code was written and tested against a particular library, have we documented that fact?
• Can this code freeze/hang/crash a running daemon? Can it stall a state run?  Are there infinite loops? Are appropriate timeouts implemented?
• Is the function interface well documented? If argument types can not be inferred by introspection, are they documented?
• Are resources such as file-handles cleaned-up after they are used?
• Is it possible that a reference-cycle exists between objects that will leak memory?
• Has the code been linted and does it pass all tests?
• Does the change fully address the problem or is it limited to a small surface area? By this, I mean that it should be clear that the submitter has looked for other cases in the function or module where the given case might also be addressed. If additional changes are necessary are they documented in the code as a FIXME or the PR and in Github as an issue to be tracked?
• Will the code throw errors/warnings/stacktraces to the console during normal operation?
• Has all the debugging been removed?
• Does the code log any sensitive data? Does it show sensitive data in process lists? Does it store sensitive data to disk and if so, does it do so in a secure manner? Are there potential race conditions in between writing the data to disk and setting the appropriate permissions?
• Is it clear from the solution that the problem is well-understood? How can somebody who has never seen the problem feel confident that this proposed change is the best one?
• What's hard-coded that might not need to be? Are we making sensible decisions for the user and allowing them to tune and change things where appropriate?
• Are utility functions used where appropriate? Does this change re-implement something we already have code for?
• Is the right thing being fixed? There are cases where it's appropriate to fix a test and cases where it's appropriate to fix the code that's under test.  Which is best for the user? Is this change a shortcut or a solution that will be solid in the months and years to come?
• How will this code react to changes elsewhere in the code base? What is it coupled to and have we fully thought through how best to present a coherent interface to consumers of a given function or method?
• Does this PR try to fix too many bugs/problems at once?
• Should this be split into multiple PRs to make them easier to test and reason about?
Pull Request Requirements The following outlines what is required before a pull request can be merged into the salt project. For each of these requirements, an exception can be made that requires 3 approvals before merge. The exceptions are detailed more below.
All PR requirements • Approval Required: approval review from core team member OR 1 approval review from captain of working group
• Cannot merge your own PR until 1 reviewer approves from defined list above that is not the author.
• All Tests Pass
Bug Fix PR requirements • Test Coverage: regression test written to cover bug fix. Contributors only need to write test coverage for their specific changes.
• Point to the issue the PR is resolving. If there is not an issue one will need to be created.
Feature PR requirements • Test Coverage: tests written to cover new feature. Contributors only need to write test coverage for their specific changes.
• Release Notes: Add note in release notes of new feature for relative release.
• Add .. versionadded:: <release> to module's documentation. If you are not certain which release your fix will be included in you can include TBD and the PR reviewer will let you know the correct name of the release you need to update to the versionadded.
Exceptions to all requirements As previously stated, all of the above requirements can be bypassed with 3 approvals.  PR's that do not require tests include:
• cosmetic changes (for example changing from log.debug to log.trace)
• fixing tests
• pylint
• changes outside of the salt directory
Reporting Bugs Salt uses GitHub to track open issues and feature requests.
To file a bug, please navigate to the new issue page for the Salt project.
In an issue report, please include the following information:
• The output of salt --versions-report from the relevant machines. This can also be gathered remotely by using salt <my_tgt> test.versions_report.
• A description of the problem including steps taken to cause the issue to occur and the expected behaviour.
• Any steps taken to attempt to remediate the problem.
• Any configuration options set in a configuration file that may be relevant.
• A reproduceable test case. This may be as simple as an SLS file that illustrates a problem or it may be a link to a repository that contains a number of SLS files that can be used together to re-produce a problem. If the problem is transitory, any information that can be used to try and reproduce the problem is helpful.
• [Optional] The output of each salt component (master/minion/CLI) running with the -ldebug flag set.
NOTE: Please be certain to scrub any logs or SLS files for sensitive data!
Salt Topology Salt is based on a powerful, asynchronous, network topology using ZeroMQ. Many ZeroMQ systems are in place to enable communication. The central idea is to have the fastest communication possible.
Servers The Salt Master runs 2 network services. First is the ZeroMQ PUB system. This service by default runs on port 4505/tcp and can be configured via the publish_port option in the master configuration.
Second is the ZeroMQ REP system. This is a separate interface used for all bi-directional communication with minions. By default this system binds to port 4506/tcp and can be configured via the ret_port option in the master.
PUB/SUB The commands sent out via the salt client are broadcast out to the minions via ZeroMQ PUB/SUB. This is done by allowing the minions to maintain a connection back to the Salt Master and then all connections are informed to download the command data at once. The command data is kept extremely small (usually less than 1K) so it is not a burden on the network.
Return The PUB/SUB system is a one way communication, so once a publish is sent out the PUB interface on the master has no further communication with the minion.  The minion, after running the command, then sends the command's return data back to the master via the ret_port.
Developing Salt Tutorial This tutorial assumes you have:
• a web browser
• a GitHub account (<my_account>)
• a command line (CLI)
• git
• a text editor
Fork In your browser, navigate to the saltstack/salt GitHub repository.
Click on Fork (https://github.com/saltstack/salt/#fork-destination-box).
NOTE: If you have more than one GitHub presence, for example if you are a member of a team, GitHub will ask you into which area to clone Salt. If you don't know where, then select your personal GitHub account.
Clone In your CLI, navigate to the directory into which you want clone the Salt codebase and submit the following command:
$ git clone https://github.com/<my_account>/salt.git
where <my_account> is the name of your GitHub account.  After the clone has completed, add SaltStack as a second remote and fetch any changes from upstream.
$ cd salt $ git remote add upstream https://github.com/saltstack/salt.git $ git fetch upstream
For this tutorial, we will be working off from the master branch, which is the default branch for the SaltStack GitHub project.  This branch needs to track upstream/master so that we will get all upstream changes when they happen.
$ git checkout master $ git branch --set-upstream-to upstream/master
Fetch Fetch any upstream changes on the master branch and sync them to your local copy of the branch with a single command:
$ git pull --rebase
NOTE: For an explanation on pull vs pull --rebase and other excellent points, see this article by Mislav Marohnić.
Branch Now we are ready to get to work.  Consult the sprint beginner bug list and select an execution module whose __virtual__ function needs to be updated.  I'll select the alternatives module.
Create a new branch off from master.  Be sure to name it something short and descriptive.
$ git checkout -b virt_ret
Edit Edit the file you have selected, and verify that the changes are correct.
$ vim salt/modules/alternatives.py $ git diff
diff --git a/salt/modules/alternatives.py b/salt/modules/alternatives.py index 1653e5f..30c0a59 100644 --- a/salt/modules/alternatives.py +++ b/salt/modules/alternatives.py @@ -30,7 +30,7 @@ def __virtual__(): ''' if os.path.isdir('/etc/alternatives'): return True -        return False +        return (False, 'Cannot load alternatives module: /etc/alternatives dir not found')
def _get_cmd():
Commit Stage and commit the changes.  Write a descriptive commit summary, but try to keep it less than 50 characters.  Review your commit.
$ git add salt/modules/alternatives.py $ git commit -m 'modules.alternatives: __virtual__ return err msg' $ git show
NOTE: If you need more room to describe the changes in your commit, run git commit (without the -m, message, option) and you will be presented with an editor.  The first line is the commit summary and should still be 50 characters or less.  The following paragraphs you create are free form and will be preserved as part of the commit.
Push Push your branch to your GitHub account.  You will likely need to enter your GitHub username and password.
$ git push origin virt_ret Username for 'https://github.com': <my_account> Password for 'https://<my_account>@github.com':
NOTE: If authentication over https does not work, you can alternatively setup ssh keys.  Once you have done this, you may need add the keys to your git repository configuration
$ git config ssh.key ~/.ssh/<key_name>
where <key_name> is the file name of the private key you created.
Merge In your browser, navigate to the new pull request page on the saltstack/salt GitHub repository and click on compare across forks. Select <my_account> from the list of head forks and the branch you are wanting to merge into master (virt_ret in this case).
When you have finished reviewing the changes, click Create pull request.
If your pull request contains only a single commit, the title and comment will be taken from that commit's summary and message, otherwise the branch name is used for the title.  Edit these fields as necessary  and click Create pull request.
NOTE: Although these instructions seem to be the official pull request procedure on github's website, here are two alternative methods that are simpler.
• If you navigate to your clone of salt, https://github.com/<my_account>/salt, depending on how old your branch is or how recently you pushed updates on it, you may be presented with a button to create a pull request with your branch.
• I find it easiest to edit the following URL:
https://github.com/saltstack/salt/compare/master...<my_account>:virt_ret
Resources GitHub offers many great tutorials on various aspects of the git- and GitHub-centric development workflow:
https://help.github.com/
There are many topics covered by the Salt Developer documentation:
https://docs.saltproject.io/en/latest/topics/development/index.html
The contributing documentation presents more details on specific contributing topics:
https://docs.saltproject.io/en/latest/topics/development/contributing.html