neulab/tldr huggingface dataset:
More advanced HTTP features Load cookies $ aria2c --load-cookies=cookies.txt "http://host/file.zip"
NOTE: You can use Firefox/Mozilla/Chromium's cookie files without modification.
Resume download started by web browsers or other programs $ aria2c -c -s2 "http://host/partiallydownloadedfile.zip"
NOTE: This will only work when the initial download was not multi-segmented.
Client certificate authorization for SSL/TLS Specify a PKCS12 file as follows:
$ aria2c --certificate=/path/to/mycert.p12
NOTE: The file specified in --certificate must be contain one PKCS12 encoded certificate and key. The password must be blank.
Alternatively, if PEM files are supported, use a command like the following:
$ aria2c --certificate=/path/to/mycert.pem --private-key=/path/to/mykey.pem https://host/file
NOTE: The file specified in --private-key must be decrypted. The behavior when encrypted one is given is undefined.
Verify SSL/TLS servers using given CA certificates $ aria2c --ca-certificate=/path/to/ca-certificates.crt --check-certificate https://host/file
NOTE: This option is only available when aria2 was compiled against GnuTLS or OpenSSL.  WinTLS and AppleTLS will always use the system certificate store. Instead of `--ca-certificate install the certificate in that store.
RPC Encrypt RPC traffic with SSL/TLS Specify a server PKC12 file:
$ aria2c --enable-rpc --rpc-certificate=/path/to/server.p12 --rpc-secure
NOTE: The file specified in --rpc-certificate must be contain one PKCS12 encoded certificate and key. The password must be blank.
Alternatively, when PEM files are supported (GnuTLS and OpenSSL), specify the server certificate file and private key file as follows:
$ aria2c --enable-rpc --rpc-certificate=/path/to/server.crt --rpc-private-key=/path/to/server.key --rpc-secure
And more advanced features Throttle the download speed Per-download:
$ aria2c --max-download-limit=100K file.metalink
$ aria2c --max-overall-download-limit=100K file.metalink
Repair a damaged download $ aria2c -V file.metalink
NOTE: Repairing damaged downloads can be done efficiently when used with BitTorrent or Metalink with chunk checksums.
Drop connections if download speed is lower than a specified limit $ aria2c --lowest-speed-limit=10K file.metalink
Parameterized URI support You can specify set of parts:
$ aria2c -P "http://{host1,host2,host3}/file.iso"
You can specify numeric sequence:
$ aria2c -Z -P "http://host/image[000-100].png"
NOTE: The -Z option is required if the URIs don't point to the same file, such as in the above example.
You can specify step counter:
$ aria2c -Z -P "http://host/image[A-Z:2].png"
Verifying checksums $ aria2c --checksum=sha-1=0192ba11326fe2298c8cb4de616f4d4140213837 http://example.org/file
Parallel downloads of an arbitrary number of URIs, metalink, torrent $ aria2c -j3 -Z "http://host/file1" file2.torrent file3.metalink
BitTorrent Encryption Encrypt the whole payload using ARC4 (obfuscation):
$ aria2c --bt-min-crypto-level=arc4 --bt-require-crypto=true file.torrent
Project Web Site: https://aria2.github.io/
Metalink Homepage: http://www.metalinker.org/
The Metalink Download Description Format: RFC 5854
Copyright (C) 2006, 2015 Tatsuhiro Tsujikawa
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
In addition, as a special exception, the copyright holders give permission to link the code of portions of this program with the OpenSSL library under certain conditions as described in each individual source file, and distribute linked combinations including the two.  You must obey the GNU General Public License in all respects for all of the code used other than OpenSSL.  If you modify file(s) with this exception, you may extend this exception to your version of the file(s), but you are not obligated to do so.  If you do not wish to do so, delete this exception statement from your version.  If you delete this exception statement from all source files in the program, then also delete it here.
READELF(1)                    GNU Development Tools                   READELF(1)
readelf - display information about ELF files
readelf [-a|--all] [-h|--file-header] [-l|--program-headers|--segments] [-S|--section-headers|--sections] [-g|--section-groups] [-t|--section-details] [-e|--headers] [-s|--syms|--symbols] [--dyn-syms|--lto-syms] [--demangle=style|--no-demangle] [--recurse-limit|--no-recurse-limit] [-n|--notes] [-r|--relocs] [-u|--unwind] [-d|--dynamic] [-V|--version-info] [-A|--arch-specific] [-D|--use-dynamic] [-L|--lint|--enable-checks] [-x <number or name>|--hex-dump=<number or name>] [-p <number or name>|--string-dump=<number or name>] [-R <number or name>|--relocated-dump=<number or name>] [-z|--decompress] [-c|--archive-index] [-w[lLiaprmfFsoORtUuTgAckK]| --debug-dump[=rawline,=decodedline,=info,=abbrev,=pubnames,=aranges,=macro,=frames,=frames-interp,=str,=str-offsets,=loc,=Ranges,=pubtypes,=trace_info,=trace_abbrev,=trace_aranges,=gdb_index,=addr,=cu_index,=links,=follow-links]] [--dwarf-depth=n] [--dwarf-start=n] [--ctf=section] [--ctf-parent=section] [--ctf-symbols=section] [--ctf-strings=section] [-I|--histogram] [-v|--version] [-W|--wide] [-T|--silent-truncation] [-H|--help] elffile...
readelf displays information about one or more ELF format object files. The options control what particular information to display.
elffile... are the object files to be examined.  32-bit and 64-bit ELF files are supported, as are archives containing ELF files.
This program performs a similar function to objdump but it goes into more detail and it exists independently of the BFD library, so if there is a bug in BFD then readelf will not be affected.
The long and short forms of options, shown here as alternatives, are equivalent.  At least one option besides -v or -H must be given.
-a --all Equivalent to specifying --file-header, --program-headers, --sections, --symbols, --relocs, --dynamic, --notes, --version-info, --arch-specific, --unwind, --section-groups and --histogram.
Note - this option does not enable --use-dynamic itself, so if that option is not present on the command line then dynamic symbols and dynamic relocs will not be displayed.
-h --file-header Displays the information contained in the ELF header at the start of the file.
-l --program-headers --segments Displays the information contained in the file's segment headers, if it has any.
-S --sections --section-headers Displays the information contained in the file's section headers, if it has any.
-g --section-groups Displays the information contained in the file's section groups, if it has any.
-t --section-details Displays the detailed section information. Implies -S.
-s --symbols --syms Displays the entries in symbol table section of the file, if it has one.  If a symbol has version information associated with it then this is displayed as well.  The version string is displayed as a suffix to the symbol name, preceded by an @ character.  For example foo@VER_1.  If the version is the default version to be used when resolving unversioned references to the symbol then it is displayed as a suffix preceded by two @ characters.  For example foo@@VER_2.
--dyn-syms Displays the entries in dynamic symbol table section of the file, if it has one.  The output format is the same as the format used by the --syms option.
--lto-syms Displays the contents of any LTO symbol tables in the file.
-C --demangle[=style] Decode (demangle) low-level symbol names into user-level names.  This makes C++ function names readable.  Different compilers have different mangling styles.  The optional demangling style argument can be used to choose an appropriate demangling style for your compiler.
--no-demangle Do not demangle low-level symbol names.  This is the default.
-e --headers Display all the headers in the file.  Equivalent to -h -l -S.
-n --notes Displays the contents of the NOTE segments and/or sections, if any.
-r --relocs Displays the contents of the file's relocation section, if it has one.
-u --unwind Displays the contents of the file's unwind section, if it has one. Only the unwind sections for IA64 ELF files, as well as ARM unwind tables (".ARM.exidx" / ".ARM.extab") are currently supported.  If support is not yet implemented for your architecture you could try dumping the contents of the .eh_frames section using the --debug-dump=frames or --debug-dump=frames-interp options.
-d --dynamic Displays the contents of the file's dynamic section, if it has one.
-V --version-info Displays the contents of the version sections in the file, it they exist.
-A --arch-specific Displays architecture-specific information in the file, if there is any.
-D --use-dynamic When displaying symbols, this option makes readelf use the symbol hash tables in the file's dynamic section, rather than the symbol table sections.
When displaying relocations, this option makes readelf display the dynamic relocations rather than the static relocations.
-L --lint --enable-checks Displays warning messages about possible problems with the file(s) being examined.  If used on its own then all of the contents of the file(s) will be examined.  If used with one of the dumping options then the warning messages will only be produced for the things being displayed.
-x <number or name> --hex-dump=<number or name> Displays the contents of the indicated section as a hexadecimal bytes.  A number identifies a particular section by index in the section table; any other string identifies all sections with that name in the object file.
-R <number or name> --relocated-dump=<number or name> Displays the contents of the indicated section as a hexadecimal bytes.  A number identifies a particular section by index in the section table; any other string identifies all sections with that name in the object file.  The contents of the section will be relocated before they are displayed.
-p <number or name> --string-dump=<number or name> Displays the contents of the indicated section as printable strings. A number identifies a particular section by index in the section table; any other string identifies all sections with that name in the object file.
-z --decompress Requests that the section(s) being dumped by x, R or p options are decompressed before being displayed.  If the section(s) are not compressed then they are displayed as is.
-c --archive-index Displays the file symbol index information contained in the header part of binary archives.  Performs the same function as the t command to ar, but without using the BFD library.
-w[lLiaprmfFsOoRtUuTgAckK] --debug-dump[=rawline,=decodedline,=info,=abbrev,=pubnames,=aranges,=macro,=frames,=frames-interp,=str,=str-offsets,=loc,=Ranges,=pubtypes,=trace_info,=trace_abbrev,=trace_aranges,=gdb_index,=addr,=cu_index,=links,=follow-links] Displays the contents of the DWARF debug sections in the file, if any are present.  Compressed debug sections are automatically decompressed (temporarily) before they are displayed.  If one or more of the optional letters or words follows the switch then only those type(s) of data will be dumped.  The letters and words refer to the following information:
--ctf-symbols=section --ctf-strings=section Specify the name of another section from which the CTF file can inherit strings and symbols.  By default, the ".symtab" and its linked string table are used.
If either of --ctf-symbols or --ctf-strings is specified, the other must be specified as well.
-I --histogram Display a histogram of bucket list lengths when displaying the contents of the symbol tables.
-v --version Display the version number of readelf.
-W --wide Don't break output lines to fit into 80 columns. By default readelf breaks section header and segment listing lines for 64-bit ELF files, so that they fit into 80 columns. This option causes readelf to print each section header resp. each segment one a single line, which is far more readable on terminals wider than 80 columns.
-T --silent-truncation Normally when readelf is displaying a symbol name, and it has to truncate the name to fit into an 80 column display, it will add a suffix of "[...]" to the name.  This command line option disables this behaviour, allowing 5 more characters of the name to be displayed and restoring the old behaviour of readelf (prior to release 2.35).
-H --help Display the command-line options understood by readelf.
objdump(1), and the Info entries for binutils.
LVM(8)                       System Manager's Manual                      LVM(8)
lvm - Linux Logical Volume Manager
lvm is a logical volume manager for Linux.  It enables you to concatenate several physical volumes (hard disks etc.)  into a so called volume group (VG, see pvcreate(8) and vgcreate(8) ) forming a storage pool, like a virtual disk.  IDE, SCSI disks as well as multiple devices (MD) are supported.  The storage capacity of a volume group can be divided into logical volumes (LVs), like virtual disk partitions. The size of a logical volume is in multiples of physical extents (PEs, see lvcreate(8) ). The size of the physical extents can be configured at volume group creation time. If a logical volume is too small or too large you can change its size at runtime ( see lvextend(8) and lvreduce(8) ). lvcreate(8) can be used to create snapshots of existing logical volumes (so called original logical volumes in this context) as well. Creating a snapshot logical volumes grants access to the contents of the original logical volume it is associated with and exposes the read only contents at the creation time of the snapshot. This is useful for backups or for keeping several versions of filesystems online. If you run out of space in a volume group it is possible to add one or more pvcreate'd disks to the system and put them into an existing volume group ( see vgextend(8) ). The space on these new physical volumes can be dynamically added to logical volumes in that volume group ( see lvextend(8) ). To remove a physical volume from the system you can move allocated logical extents to different physical volumes ( see pvmove(8) ). After the pvmove the volume group can be reduced with the vgreduce(8) command. Inactive volume groups must be activated with vgchange(8) before use. vgcreate(8) automatically activates a newly created volume group.
Abbreviations PV for physical volume, PE for physical extent, VG for volume group, LV for logical volume, and LE for logical extent.
Command naming convention All command names corresponding to physical volumes start with pv, all the ones concerned with volume groups start with vg and all for logical volumes with lv.  General purpose commands for the lvm as a whole start with lvm.
The volume group descriptor area (or VGDA for short) holds the necessary metadata to handle the LVM functionality. It is stored at the beginning of each pvcreate'd disk.  It contains four parts: one PV descriptor, one VG descriptor, the LV descriptors and several PE descriptors. LE descriptors are derived from the PE ones at vgchange(8) time. Automatic backups of the VGDA are stored in files in /etc/lvmconf/ (please see vgcfgbackup(8)/vgcfgrestore(8) too). Take care to include these files in your regular (tape) backups as well.
Limits Currently up to 99 volume groups with a grand total of 256 logical volumes can be created. The limit for the logical volumes is not caused by the LVM but by Linux 8 bit device minor numbers.
This means that you can have 99 volume groups with 1-3 logical volumes each or on the other hand 1 volume group with up to 256 logical volumes or anything in between these extreme examples.
Depending on the physical extent size specified at volume group creation time (see vgcreate(8) ), logical volumes of between a maximum of 512 Megabytes and 1 Petabyte can be created.  Actual Linux kernels on IA32 limit these lvm possibilities to a maximum of 2 Terabytes per logical and per physical volume as well. This enables you to have as much as 256 Terabytes under LVM control with all possible 128 scsi disk subsystems. You can have up to 65534 logical extents (on IA32) in a logical volume at the cost of 1 Megabyte in kernel memory.  Physical volumes can have up to 65534 physical extents.
/proc filesystem support The operational state of active volume groups with their physical and logical volumes can be found in the /proc/lvm/ directory. /proc/lvm/global contains a summary of all available information regarding all VGs, LVs and PVs.  The two flags for PV status in brackets mean A/I for active/inactive and A/N for allocatable or non-allocatable. The four flags for LV status in brackets mean A/I for active/inactive, R/W for read-only or read/write, D/C for discontiguous or contiguous and L/S for linear or striped.  S can optionally be followed by the number of stripes in the set.  At /proc/lvm/VGs/ starts a subdirectory hierarchy containing information about every VG in a different subdirectory named /proc/lvm/VGs/VolumeGroupName where VolumeGroupName stands for an arbitrary VG name.  /proc/lvm/VGs/VolumeGroupName/ in turn holds a file group containing summary information for the VG as a total. /proc/lvm/VGs/VolumeGroupName/LVs/LogicalVolumeName holds information for an arbitrary LV named LogicalVolumeName /proc/lvm/VGs/VolumeGroupName/PVs/PhysicalVolumeName contains information for an arbitrary PV named PhysicalVolumeName.  All of the information in the files below /proc/lvm/VGs/ is presented in attribute/value pairs to be easyly parsable.
Examples We have disk partitions /dev/sda3, /dev/sdb1 and /dev/hda2 free for use and want to create a volume group named "test_vg".  Steps required:
1. Change partition type for these 3 partitions to 0x8e with fdisk.  (see pvcreate(8): 0x8e identifies LVM partitions)
2. pvcreate /dev/sda3 /dev/sdb1 /dev/hda2
3. vgcreate test_vg /dev/sda3 /dev/sdb1 /dev/hda2
With our volume group "test_vg" now online, we can create logical volumes. For example a logical volume with a size of 100MB and standard name (/dev/test_vg/lvol1) and another one named "my_test_lv" with size 200MB striped (RAID0) across all the three physical volumes.
Steps required:
1. lvcreate -L 100 test_vg
2. lvcreate -L 200 -n my_test_lv -i 3 test_vg
Now let's rock and roll.  For example create a file system with "mkfs -t ext2 /dev/test_vg/my_test_lv" and mount it with "mount /dev/test_vg/my_test_lv /usr1"
See also e2fsadm(8), lvchange(8), lvcreate(8), lvdisplay(8), lvextend(8), lvmchange(8), lvmdiskscan(8), lvmcreate_initrd(8), lvmsadc(8), lvmsar(8), lvreduce(8), lvremove(8), lvrename(8), lvscan(8), pvchange(8), pvcreate(8), pvdata(8), pvdisplay(8), pvmove(8), pvscan(8), vgcfgbackup(8), vgcfgrestore(8), vgchange(8), vgck(8), vgcreate(8), vgdisplay(8), vgexport(8), vgextend(8), vgimport(8), vgmerge(8), vgmknodes(8), vgreduce(8), vgremove(8), vgrename(8), vgscan(8), vgsplit(8)
TOX(1)                                 tox                                TOX(1)
tox - tox Documentation
VISION: STANDARDIZE TESTING IN PYTHON tox aims to automate and standardize testing in Python.  It is part of a larger vision of easing the packaging, testing and release process of Python software.
WHAT IS TOX? tox is a generic virtualenv management and test command line tool you can use for:
• checking your package installs correctly with different Python versions and interpreters
• running your tests in each of the environments, configuring your test tool of choice
• acting as a frontend to Continuous Integration servers, greatly reducing boilerplate and merging CI and shell-based testing.
First, install tox with pip install tox.  Then put basic information about your project and the test environments you want your project to run in into a tox.ini file residing right next to your setup.py file:
# content of: tox.ini , put in same dir as setup.py [tox] envlist = py27,py36
[testenv] # install pytest in the virtualenv where commands will be executed deps = pytest commands = # NOTE: you can run any command line tool here - not just tests pytest
You can also try generating a tox.ini file automatically, by running tox-quickstart and then answering a few simple questions.
To sdist-package, install and test your project against Python2.7 and Python3.6, just type:
and watch things happening (you must have python2.7 and python3.6 installed in your environment otherwise you will see errors).  When you run tox a second time you'll note that it runs much faster because it keeps track of virtualenv details and will not recreate or re-install dependencies.  You also might want to checkout examples to get some more ideas.
[image] tox workflow diagram.UNINDENT
tox roughly follows the following phases:
1. configuration: load tox.ini and merge it with options from the command line and the operating system environment variables.
2. packaging (optional): create a source distribution of the current project by invoking
Note that for this operation the same Python environment will be used as the one tox is installed into (therefore you need to make sure that it contains your build dependencies).  Skip this step for application projects that don't have a setup.py.
3. environment - for each tox environment (e.g. py27, py36) do: 1. environment creation: create a fresh environment, by default virtualenv is used. tox will automatically try to discover a valid Python interpreter version by using the environment name (e.g. py27 means Python 2.7 and the basepython configuration value) and the current operating system PATH value. This is created at first run only to be re-used at subsequent runs. If certain aspects of the project change, a re-creation of the environment is automatically triggered. To force the recreation tox can be invoked with -r/--recreate.
2. install (optional): install the environment dependencies specified inside the deps configuration section, and then the earlier packaged source distribution.  By default pip is used to install packages, however one can customise this via install_command. Note pip will not update project dependencies (specified either in the install_requires or the extras section of the setup.py) if any version already exists in the virtual environment; therefore we recommend to recreate your environments whenever your project dependencies change.
3. commands: run the specified commands in the specified order. Whenever the exit code of any of them is not zero stop, and mark the environment failed. Note, starting a command with a single dash character means ignore exit code.
6. report print out a report of outcomes for each tox environment:
____________________ summary ____________________ py27: commands succeeded ERROR:   py36: commands failed
Only if all environments ran successfully tox will return exit code 0 (success). In this case you'll also see the message congratulations :).
tox will take care of environment isolation for you: it will strip away all operating system environment variables not specified via passenv. Furthermore, it will also alter the PATH variable so that your commands resolve first and foremost within the current active tox environment. In general all executables in the path are available in commands, but tox will emit a warning if it was not explicitly allowed via whitelist_externals.
• automation of tedious Python related test activities
• test your Python package against many interpreter and dependency configs
• automatic customizable (re)creation of virtualenv test environments
• installs your setup.py based project into each virtual environment
• test-tool agnostic: runs pytest, nose or unittests in a uniform manner
• plugin system to modify tox execution with simple hooks.
• uses pip and setuptools by default.  Support for configuring the installer command through install_command=ARGV.
• cross-Python compatible: CPython-2.7, 3.4 and higher, Jython and pypy.
• cross-platform: Windows and Unix style environments
• integrates with continuous integration servers like Jenkins (formerly known as Hudson) and helps you to avoid boilerplatish and platform-specific build-step hacks.
• full interoperability with devpi: is integrated with and is used for testing in the devpi system, a versatile PyPI index server and release managing tool.
• driven by a simple ini-style config file
• documented examples and configuration
• concise reporting about tool invocations and configuration errors
• professionally supported
• supports using different / multiple PyPI index servers
tox has influenced several other projects in the Python test automation space. If tox doesn't quite fit your needs or you want to do more research, we recommend taking a look at these projects:
• Invoke is a general-purpose task execution library, similar to Make. Invoke is far more general-purpose than tox but it does not contain the Python testing-specific features that tox specializes in.
• Nox is a project similar in spirit to tox but different in approach. Nox's key difference is that it uses Python scripts instead of a configuration file. Nox might be useful if you find tox's configuration too limiting but aren't looking to move to something as general-purpose as Invoke or Make.
tox installation Install info in a nutshell Pythons: CPython 2.7 and 3.4 or later, Jython-2.5.1, pypy-1.9ff
Operating systems: Linux, Windows, OSX, Unix
Installer Requirements: setuptools
License: MIT license
git repository: https://github.com/tox-dev/tox
Installation with pip Use the following command:
It is fine to install tox itself into a virtualenv environment.
Install from clone Consult the GitHub page how to clone the git repository: https://github.com/tox-dev/tox
and then install in your environment with something like:
$ cd <path/to/clone> $ pip install .
or install it editable if you want code changes to propagate automatically:
$ cd <path/to/clone> $ pip install --editable .
so that you can do changes and submit patches.
[Linux/macOS] Install via your package manager You can also find tox packaged for many Linux distributions and Homebrew for macOs - usually under the name of python-tox or simply tox. Be aware though that there also other projects under the same name (most prominently a secure chat client with no affiliation to this project), so make sure you install the correct package.
tox configuration and usage examples Basic usage a simple tox.ini / default environments Put basic information about your project and the test environments you want your project to run in into a tox.ini file that should reside next to your setup.py file:
# content of: tox.ini , put in same dir as setup.py [tox] envlist = py27,py36 [testenv] # install testing framework # ... or install anything else you might need here deps = pytest # run the tests # ... or run any other command line tool you need to run here commands = pytest
To sdist-package, install and test your project, you can now type at the command prompt:
This will sdist-package your current project, create two virtualenv Environments, install the sdist-package into the environments and run the specified command in each of them.  With:
tox -e py36
you can run restrict the test run to the python3.6 environment.
Available "default" test environments names are:
py py2 py27 py3 py34 py35 py36 py37 py38 jython pypy pypy2 pypy27 pypy3 pypy35
The environment py uses the version of Python used to invoke tox.
However, you can also create your own test environment names, see some of the examples in examples.
pyproject.toml tox legacy ini The tox configuration can also be in pyproject.toml (if you want to avoid an extra file).
Currently only the old format is supported via legacy_tox_ini, a native implementation is planned though.
[build-system] requires = [ "setuptools >= 35.0.2", "wheel >= 0.29.0"] build-backend = "setuptools.build_meta"
[tool.tox] legacy_tox_ini = """ [tox] envlist = py27,py36
[testenv] deps = pytest >= 3.0.0, <4 commands = pytest """
Note that when you define a pyproject.toml you must define the build-requires section per PEP-518.
specifying a platform New in version 2.0.
If you want to specify which platform(s) your test environment runs on you can set a platform regular expression like this:
[testenv] platform = linux2|darwin
If the expression does not match against sys.platform the test environment will be skipped.
whitelisting non-virtualenv commands New in version 1.5.
Sometimes you may want to use tools not contained in your virtualenv such as make, bash or others. To avoid warnings you can use the whitelist_externals testenv configuration:
# content of tox.ini [testenv] whitelist_externals = make /bin/bash
depending on requirements.txt or defining constraints New in version 1.6.1.
(experimental) If you have a requirements.txt file or a constraints.txt file you can add it to your deps variable like this:
[testenv] deps = -rrequirements.txt
[testenv] deps = -cconstraints.txt
[testenv] deps = -rrequirements.txt -cconstraints.txt
All installation commands are executed using {toxinidir} (the directory where tox.ini resides) as the current working directory.  Therefore, the underlying pip installation will assume requirements.txt or constraints.txt to exist at {toxinidir}/requirements.txt or {toxinidir}/constraints.txt.
This is actually a side effect that all elements of the dependency list is directly passed to pip.
For more details on requirements.txt files or constraints.txt files please see:
• https://pip.pypa.io/en/stable/user_guide/#requirements-files
• https://pip.pypa.io/en/stable/user_guide/#constraints-files
using a different default PyPI url New in version 0.9.
To install dependencies and packages from a different default PyPI server you can type interactively:
tox -i https://pypi.my-alternative-index.org
This causes tox to install dependencies and the sdist install step to use the specified url as the index server.
You can cause the same effect by this tox.ini content:
[tox] indexserver = default = https://pypi.my-alternative-index.org
installing dependencies from multiple PyPI servers New in version 0.9.
You can instrument tox to install dependencies from different PyPI servers, example:
[tox] indexserver = DEV = https://mypypiserver.org
[testenv] deps = # docutils will be installed directly from PyPI docutils # mypackage will be installed from custom "DEV" PyPI url :DEV:mypackage
This configuration will install docutils from the default Python PYPI server and will install the mypackage from our DEV indexserver, and the respective https://mypypiserver.org url.  You can override config file settings from the command line like this:
tox -i DEV=https://pypi.org/simple  # changes :DEV: package URLs tox -i https://pypi.org/simple      # changes default
further customizing installation New in version 1.6.
By default tox uses pip to install packages, both the package-under-test and any dependencies you specify in tox.ini.  You can fully customize tox's install-command through the testenv-specific install_command=ARGV setting.  For instance, to use pip's --find-links and --no-index options to specify an alternative source for your dependencies:
[testenv] install_command = pip install --pre --find-links https://packages.example.com --no-index {opts} {packages}
forcing re-creation of virtual environments New in version 0.9.
To force tox to recreate a (particular) virtual environment:
tox --recreate -e py27
would trigger a complete reinstallation of the existing py27 environment (or create it afresh if it doesn't exist).
passing down environment variables New in version 2.0.
By default tox will only pass the PATH environment variable (and on windows SYSTEMROOT and PATHEXT) from the tox invocation to the test environments.  If you want to pass down additional environment variables you can use the passenv option:
[testenv] passenv = LANG
When your test commands execute they will execute with the same LANG setting as the one with which tox was invoked.
setting environment variables New in version 1.0.
If you need to set an environment variable like PYTHONPATH you can use the setenv directive:
[testenv] setenv = PYTHONPATH = {toxinidir}/subdir
When your test commands execute they will execute with a PYTHONPATH setting that will lead Python to also import from the subdir below the directory where your tox.ini file resides.
special handling of PYTHONHASHSEED New in version 1.6.2.
By default, tox sets PYTHONHASHSEED for test commands to a random integer generated when tox is invoked.  This mimics Python's hash randomization enabled by default starting in Python 3.3.  To aid in reproducing test failures, tox displays the value of PYTHONHASHSEED in the test output.
You can tell tox to use an explicit hash seed value via the --hashseed command-line option to tox.  You can also override the hash seed value per test environment in tox.ini as follows:
[testenv] setenv = PYTHONHASHSEED = 100
If you wish to disable this feature, you can pass the command line option --hashseed=noset when tox is invoked. You can also disable it from the tox.ini by setting PYTHONHASHSEED = 0 as described above.
Integration with setup.py test command WARNING: Integrating tox with setup.py test is as of October 2016 discouraged as it breaks packaging/testing approaches used by downstream distributions which expect setup.py test to run tests with the invocation interpreter rather than setting up many virtualenvs and installing packages.  If you need to define setup.py test, you can see how to integrate your eventual test runner with it, here is an example of setup.py test integration with pytest.  As the python eco-system rather moves away from using setup.py as a tool entry point it's maybe best to not go for any setup.py test integration.
Ignoring a command exit code In some cases, you may want to ignore a command exit code. For example:
[testenv:py27] commands = coverage erase {envbindir}/python setup.py develop coverage run -p setup.py test coverage combine - coverage html {envbindir}/flake8 loads
By using the - prefix, similar to a make recipe line, you can ignore the exit code for that command.
Compressing dependency matrix If you have a large matrix of dependencies, python versions and/or environments you can use generative-envlist and conditional settings to express that in a concise form:
[tox] envlist = py{27,34,36}-django{15,16}-{sqlite,mysql}
[testenv] deps = django15: Django>=1.5,<1.6 django16: Django>=1.6,<1.7 # use PyMySQL if factors "py34" and "mysql" are present in env name py34-mysql: PyMySQL # use urllib3 if any of "py36" or "py27" are present in env name py27,py36: urllib3 # mocking sqlite on 2.7 and 3.6 if factor "sqlite" is present py{27,36}-sqlite: mock
Prevent symbolic links in virtualenv By default virtualenv will use symlinks to point to the system's python files, modules, etc.  If you want the files to be copied instead, possibly because your filesystem is not capable of handling symbolic links, you can instruct virtualenv to use the "--always-copy" argument meant exactly for that purpose, by setting the alwayscopy directive in your environment:
[testenv] alwayscopy = True
Parallel mode tox allows running environments in parallel:
• Invoke by using the --parallel or -p flag. After the packaging phase completes tox will run in parallel processes tox environments (spins a new instance of the tox interpreter, but passes through all host flags and environment variables).
• -p takes an argument specifying the degree of parallelization:
• all to run all invoked environments in parallel,
• auto to limit it to CPU count,
• or pass an integer to set that limit.
• Parallel mode displays a progress spinner while running tox environments in parallel, and reports outcome of these as soon as completed with a human readable duration timing attached.
• Parallel mode by default shows output only of failed environments and ones marked as parallel_show_output =True.
• There's now a concept of dependency between environments (specified via depends), tox will re-order the environment list to be run to satisfy these dependencies (in sequential run too). Furthermore, in parallel mode, will only schedule a tox environment to run once all of its dependencies finished (independent of their outcome).
WARNING: depends does not pull in dependencies into the run target, for example if you select py27,py36,coverage via the -e tox will only run those three (even if coverage may specify as depends other targets too - such as py27, py35, py36, py37).
• --parallel-live/-o allows showing the live output of the standard output and error, also turns off reporting described above.
• Note: parallel evaluation disables standard input. Use non parallel invocation if you need standard input.
Example final output:
$ tox -e py27,py36,coverage -p all ✔ OK py36 in 9.533 seconds ✔ OK py27 in 9.96 seconds ✔ OK coverage in 2.0 seconds ___________________________ summary ______________________________________________________ py27: commands succeeded py36: commands succeeded coverage: commands succeeded congratulations :)
Example progress bar, showing a rotating spinner, the number of environments running and their list (limited up to 120 characters):
⠹ [2] py27 | py36
packaging Although one can use tox to develop and test applications one of its most popular usage is to help library creators. Libraries need first to be packaged, so then they can be installed inside a virtual environment for testing. To help with this tox implements PEP-517 and PEP-518. This means that by default tox will build source distribution out of source trees. Before running test commands pip is used to install the source distribution inside the build environment.
To create a source distribution there are multiple tools out there and with PEP-517 and PEP-518 you can easily use your favorite one with tox. Historically tox only supported setuptools, and always used the tox host environment to build a source distribution from the source tree. This is still the default behavior.  To opt out of this behaviour you need to set isolated builds to true.
setuptools Using the pyproject.toml file at the root folder (alongside setup.py) one can specify build requirements.
[build-system] requires = [ "setuptools >= 35.0.2", "setuptools_scm >= 2.0.0, <3" ] build-backend = "setuptools.build_meta"
# tox.ini [tox] isolated_build = True
flit flit requires Python 3, however the generated source distribution can be installed under python 2. Furthermore it does not require a setup.py file as that information is also added to the pyproject.toml file.
[build-system] requires = ["flit >= 1.1"] build-backend = "flit.buildapi"
[tool.flit.metadata] module = "package_toml_flit" author = "Happy Harry" author-email = "happy@harry.com" home-page = "https://github.com/happy-harry/is"
pytest and tox It is easy to integrate pytest runs with tox.  If you encounter issues, please check if they are listed as a known issue and/or use the support channels.
Basic example Assuming the following layout:
tox.ini      # see below for content setup.py     # a classic distutils/setuptools setup.py file
and the following tox.ini content:
[tox] envlist = py35,py36
[testenv] deps = pytest               # PYPI package providing pytest commands = pytest {posargs} # substitute with tox' positional arguments
you can now invoke tox in the directory where your tox.ini resides.  tox will sdist-package your project, create two virtualenv environments with the python3.5 and python3.6 interpreters, respectively, and will then run the specified test command in each of them.
Extended example: change dir before test and use per-virtualenv tempdir Assuming the following layout:
tox.ini      # see below for content setup.py     # a classic distutils/setuptools setup.py file tests        # the directory containing tests
[testenv] changedir = tests deps = pytest # change pytest tempdir and add posargs from command line commands = pytest --basetemp={envtmpdir} {posargs}
you can invoke tox in the directory where your tox.ini resides. Differently than in the previous example the pytest command will be executed with a current working directory set to tests and the test run will use the per-virtualenv temporary directory.
Using multiple CPUs for test runs pytest supports distributing tests to multiple processes and hosts through the pytest-xdist plugin.  Here is an example configuration to make tox use this feature:
[testenv] deps = pytest-xdist changedir = tests # use three sub processes commands = pytest --basetemp={envtmpdir}  \ --confcutdir=..         \ -n 3                    \ {posargs}
Known Issues and limitations Too long filenames. you may encounter "too long filenames" for temporarily created files in your pytest run.  Try to not use the "--basetemp" parameter.
installed-versus-checkout version.  pytest collects test modules on the filesystem and then tries to import them under their fully qualified name. This means that if your test files are importable from somewhere then your pytest invocation may end up importing the package from the checkout directory rather than the installed package.
This issue may be characterised by pytest test-collection error messages, in python 3.x environments, that look like:
import file mismatch: imported module 'myproj.foo.tests.test_foo' has this __file__ attribute: /home/myuser/repos/myproj/build/lib/myproj/foo/tests/test_foo.py which is not the same as the test file we want to collect: /home/myuser/repos/myproj/myproj/foo/tests/test_foo.py HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules
There are a few ways to prevent this.
With installed tests (the tests packages are known to setup.py), a safe and explicit option is to give the explicit path {envsitepackagesdir}/mypkg to pytest.  Alternatively, it is possible to use changedir so that checked-out files are outside the import path, then pass --pyargs mypkg to pytest.
With tests that won't be installed, the simplest way to run them against your installed package is to avoid __init__.py files in test directories; pytest will still find and import them by adding their parent directory to sys.path but they won't be copied to other places or be found by Python's import system outside of pytest.
unittest2, discover and tox Running unittests with 'discover' The discover project allows to discover and run unittests and we can easily integrate it in a tox run.  As an example, perform a checkout of Pygments:
hg clone https://bitbucket.org/birkenfeld/pygments-main
and add the following tox.ini to it:
[tox] envlist = py27,py35,py36
[testenv] changedir = tests commands = discover deps = discover
If you now invoke tox you will see the creation of three virtual environments and a unittest-run performed in each of them.
Running unittest2 and sphinx tests in one go Michael Foord has contributed a tox.ini file that allows you to run all tests for his mock project, including some sphinx-based doctests.  If you checkout its repository with:
git clone https://github.com/testing-cabal/mock.git
The checkout has a tox.ini file that looks like this:
[tox] envlist = py27,py34,py35,py36
[testenv] deps = unittest2 commands = unit2 discover []
[testenv:py36] commands = unit2 discover [] sphinx-build -b doctest docs html sphinx-build docs html deps = unittest2 sphinx
[testenv:py27] commands = unit2 discover [] sphinx-build -b doctest docs html sphinx-build docs html deps = unittest2 sphinx
mock uses unittest2 to run the tests. Invoking tox starts test discovery by executing the unit2 discover commands on Python 2.7, 3.4, 3.5 and 3.6 respectively.  Against Python3.6 and Python2.7 it will additionally run sphinx-mediated doctests. If building the docs fails, due to a reST error, or any of the doctests fails, it will be reported by the tox run.
The [] parentheses in the commands provide positional substitution which means you can e.g. type:
tox -- -f -s SOMEPATH
which will ultimately invoke:
unit2 discover -f -s SOMEPATH
in each of the environments. This allows you to customize test discovery in your tox runs.
nose and tox It is easy to integrate nosetests runs with tox.  For starters here is a simple tox.ini config to configure your project for running with nose:
Basic nosetests example Assuming the following layout:
[testenv] deps = nose # ``{posargs}`` will be substituted with positional arguments from comand line commands = nosetests {posargs}
you can invoke tox in the directory where your tox.ini resides.  tox will sdist-package your project create two virtualenv environments with the python2.7 and python3.6 interpreters, respectively, and will then run the specified test command.
More examples? Also you might want to checkout general and documentation.
Generate documentation It's possible to generate the projects documentation with tox itself. The advantage of this path is that now generating the documentation can be part of the CI, and whenever any validations/checks/operations fail while generating the documentation you'll catch it within tox.
Sphinx No need to use the cryptic make file to generate a sphinx documentation. One can use tox to ensure all right dependencies are available within a virtual environment, and even specify the python version needed to perform the build. For example if the sphinx file structure is under the doc folder the following configuration will generate the documentation under {toxworkdir}/docs_out and print out a link to the generated documentation:
[testenv:docs] description = invoke sphinx-build to build the HTML docs basepython = python3.7 deps = sphinx >= 1.7.5, < 2 commands = sphinx-build -d "{toxworkdir}/docs_doctree" doc "{toxworkdir}/docs_out" --color -W -bhtml {posargs} python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'
Note here we say we also require python 3.7, allowing us to use f-strings within the sphinx conf.py. Now one can specify a separate test environment that will validate that the links are correct.
mkdocs Define one environment to write/generate the documentation, and another to deploy it. Use the config substitution logic to avoid defining dependencies multiple time:
[testenv:docs] description = Run a development server for working on documentation basepython = python3.7 deps = mkdocs >= 1.7.5, < 2 mkdocs-material commands = mkdocs build --clean python -c 'print("###### Starting local server. Press Control+C to stop server ######")' mkdocs serve -a localhost:8080
[testenv:docs-deploy] description = built fresh docs and deploy them deps = {[testenv:docs]deps} basepython = {[testenv:docs]basepython} commands = mkdocs gh-deploy --clean
General tips and tricks Interactively passing positional arguments If you invoke tox like this:
tox -- -x tests/test_something.py
the arguments after the -- will be substituted everywhere where you specify {posargs} in your test commands, for example using pytest:
[testenv] # Could also be in a specific ``[testenv:<NAME>]`` section commands = pytest {posargs}
or using nosetests:
[testenv] commands = nosetests {posargs}
the above tox invocation will trigger the test runners to stop after the first failure and to only run a particular test file.
You can specify defaults for the positional arguments using this syntax:
[testenv] commands = nosetests {posargs:--with-coverage}
Dependency changes and tracking Creating virtual environments and installing dependencies is a expensive operation.  Therefore tox tries to avoid it whenever possible, meaning it will never perform this unless it detects with absolute certainty that it needs to perform an update. A tox environment creation is made up of:
• create the virtual environment
• install dependencies specified inside deps
• if it's a library project (has build package phase), install library dependencies (with potential extras)
These three steps are only performed once (given they all succeeded). Subsequent calls that don't detect changes to the traits of that step will not alter the virtual environment in any way. When a change is detected for any of the steps, the entire virtual environment is removed and the operation starts from scratch (this is because it's very hard to determine what would the delta changes would be needed - e.g. a dependency could migrate from one dependency to another, and in this case we would need to install the new while removing the old one).
Here's what traits we track at the moment for each steps:
• virtual environment trait is tied to the python path the basepython resolves too (if this config changes, the virtual environment will be recreated),
• deps sections changes (meaning any string-level change for the entries, note requirement file content changes are not tracked),
• library dependencies are tracked at extras level (because there's no Python API to enquire about the actual dependencies in a non-tool specific way, e.g. setuptools has one way, flit something else, and poetry another).