Repository: tox-dev/tox

Issue:
Improve --notest documentation
- [x] a clear problem statement with an example

When running tox in CI there's often job step separation and failures in different kinds of stages are accounted differently. For example, in Travis CI its error/failure depending on whether the command is failing in install of script step.

- [x] suggested change with example

I'd like to see smth like `--skip-command` flag or smth like it for pre-populating env (installing deps), which I'd run in the `install` stage, and then I'd run normal tests, for which tox would reuse env created earlier.
This would help separate different logical steps of doing automatic testing in CI.

Discussion:

Patch:
diff --git a/changelog/829.doc.rst b/changelog/829.doc.rst
new file mode 100644
index 00000000..0243fb58
--- /dev/null
+++ b/changelog/829.doc.rst
@@ -0,0 +1,1 @@
+document command line interface under the config section `cli <https://tox.readthedocs.io/en/latest/config.html?highlight=cli#cli>`_ - by :user:`gaborbernat`
diff --git a/doc/_static/custom.css b/docs/_static/custom.css
similarity index 100%
rename from doc/_static/custom.css
rename to docs/_static/custom.css
diff --git a/doc/_static/img/tox.png b/docs/_static/img/tox.png
similarity index 100%
rename from doc/_static/img/tox.png
rename to docs/_static/img/tox.png
diff --git a/doc/_static/img/tox.svg b/docs/_static/img/tox.svg
similarity index 100%
rename from doc/_static/img/tox.svg
rename to docs/_static/img/tox.svg
diff --git a/doc/_static/img/toxfavi.ico b/docs/_static/img/toxfavi.ico
similarity index 100%
rename from doc/_static/img/toxfavi.ico
rename to docs/_static/img/toxfavi.ico
diff --git a/doc/_templates/localtoc.html b/docs/_templates/localtoc.html
similarity index 100%
rename from doc/_templates/localtoc.html
rename to docs/_templates/localtoc.html
diff --git a/doc/announce/changelog-only.rst b/docs/announce/changelog-only.rst
similarity index 100%
rename from doc/announce/changelog-only.rst
rename to docs/announce/changelog-only.rst
diff --git a/doc/changelog.rst b/docs/changelog.rst
similarity index 100%
rename from doc/changelog.rst
rename to docs/changelog.rst
diff --git a/doc/conf.py b/docs/conf.py
similarity index 99%
rename from doc/conf.py
rename to docs/conf.py
index 983b2402..f2a94494 100644
--- a/doc/conf.py
+++ b/docs/conf.py
@@ -13,6 +13,7 @@ extensions = [
     "sphinx.ext.extlinks",
     "sphinx.ext.intersphinx",
     "sphinx.ext.viewcode",
+    "sphinxcontrib.autoprogram",
 ]
 ROOT_SRC_TREE_DIR = Path(__file__).parents[1]
 
diff --git a/doc/config.rst b/docs/config.rst
similarity index 99%
rename from doc/config.rst
rename to docs/config.rst
index ed88bd11..72028066 100644
--- a/doc/config.rst
+++ b/docs/config.rst
@@ -869,4 +869,10 @@ Other Rules and notes
   it will be considered as relative to the ``toxinidir``, the directory
   where the configuration file resides.
 
+cli
+===
+
+.. autoprogram:: tox.cli:cli
+   :prog: tox
+
 .. include:: links.rst
diff --git a/doc/developers.rst b/docs/developers.rst
similarity index 93%
rename from doc/developers.rst
rename to docs/developers.rst
index e41697e7..7592e8e2 100644
--- a/doc/developers.rst
+++ b/docs/developers.rst
@@ -35,7 +35,7 @@ create separate folders for each Python version.
 
 For tox to find them you'll need to:
 
-- add the main installation version to the systems ``PATH`` variable (e.g. ``D:\Anaconda`` - you can use `patheditor2`_)
+- add the main installation version to the systems ``PATH`` variable (e.g. ``D:\Anaconda`` - you can use `WindowsPathEditor`_)
 - for other versions create a BAT scripts into the main installation folder to delegate the call to the correct Python
   interpreter:
 
@@ -45,7 +45,7 @@ For tox to find them you'll need to:
      REM python2.7.bat
      @D:\Anaconda\pkgs\python-2.7.13-1\python.exe %*
 
-.. _patheditor2: https://patheditor2.codeplex.com/
+.. _WindowsPathEditor: https://rix0rrr.github.io/WindowsPathEditor/
 
 This way you can also directly call from cli the matching Python version  if you need to(similarly to UNIX systems), for
 example:
diff --git a/doc/drafts/extend-envs-and-packagebuilds.md b/docs/drafts/extend-envs-and-packagebuilds.md
similarity index 100%
rename from doc/drafts/extend-envs-and-packagebuilds.md
rename to docs/drafts/extend-envs-and-packagebuilds.md
diff --git a/doc/drafts/tox_conda_notes_niccodemus.md b/docs/drafts/tox_conda_notes_niccodemus.md
similarity index 100%
rename from doc/drafts/tox_conda_notes_niccodemus.md
rename to docs/drafts/tox_conda_notes_niccodemus.md
diff --git a/doc/example/basic.rst b/docs/example/basic.rst
similarity index 100%
rename from doc/example/basic.rst
rename to docs/example/basic.rst
diff --git a/doc/example/devenv.rst b/docs/example/devenv.rst
similarity index 100%
rename from doc/example/devenv.rst
rename to docs/example/devenv.rst
diff --git a/doc/example/documentation.rst b/docs/example/documentation.rst
similarity index 100%
rename from doc/example/documentation.rst
rename to docs/example/documentation.rst
diff --git a/doc/example/general.rst b/docs/example/general.rst
similarity index 100%
rename from doc/example/general.rst
rename to docs/example/general.rst
diff --git a/doc/example/jenkins.rst b/docs/example/jenkins.rst
similarity index 100%
rename from doc/example/jenkins.rst
rename to docs/example/jenkins.rst
diff --git a/doc/example/nose.rst b/docs/example/nose.rst
similarity index 100%
rename from doc/example/nose.rst
rename to docs/example/nose.rst
diff --git a/doc/example/package.rst b/docs/example/package.rst
similarity index 100%
rename from doc/example/package.rst
rename to docs/example/package.rst
diff --git a/doc/example/platform.rst b/docs/example/platform.rst
similarity index 100%
rename from doc/example/platform.rst
rename to docs/example/platform.rst
diff --git a/doc/example/pytest.rst b/docs/example/pytest.rst
similarity index 100%
rename from doc/example/pytest.rst
rename to docs/example/pytest.rst
diff --git a/doc/example/result.rst b/docs/example/result.rst
similarity index 100%
rename from doc/example/result.rst
rename to docs/example/result.rst
diff --git a/doc/example/unittest.rst b/docs/example/unittest.rst
similarity index 100%
rename from doc/example/unittest.rst
rename to docs/example/unittest.rst
diff --git a/doc/examples.rst b/docs/examples.rst
similarity index 100%
rename from doc/examples.rst
rename to docs/examples.rst
diff --git a/doc/img/tox_flow.png b/docs/img/tox_flow.png
similarity index 100%
rename from doc/img/tox_flow.png
rename to docs/img/tox_flow.png
diff --git a/doc/index.rst b/docs/index.rst
similarity index 99%
rename from doc/index.rst
rename to docs/index.rst
index 9af287ce..5bb418ba 100644
--- a/doc/index.rst
+++ b/docs/index.rst
@@ -160,7 +160,6 @@ Current features
 
 * supports :ref:`using different / multiple PyPI index servers  <multiindex>`
 
-
 .. toctree::
    :hidden:
 
diff --git a/doc/install.rst b/docs/install.rst
similarity index 100%
rename from doc/install.rst
rename to docs/install.rst
diff --git a/doc/links.rst b/docs/links.rst
similarity index 95%
rename from doc/links.rst
rename to docs/links.rst
index df7e58f2..97789f41 100644
--- a/doc/links.rst
+++ b/docs/links.rst
@@ -20,5 +20,5 @@
 .. _mock: https://pypi.org/project/mock/
 .. _pypy: https://pypy.org
 
-.. _`Python Packaging Guide`: https://packaging.python.org/tutorials/distributing-packages/#packaging-your-project
+.. _`Python Packaging Guide`: https://packaging.python.org/tutorials/packaging-projects/
 .. _`tox.ini`: :doc:configfile
diff --git a/doc/plugins.rst b/docs/plugins.rst
similarity index 100%
rename from doc/plugins.rst
rename to docs/plugins.rst
diff --git a/doc/support.rst b/docs/support.rst
similarity index 100%
rename from doc/support.rst
rename to docs/support.rst
diff --git a/setup.py b/setup.py
index 06dd61d9..50eff205 100644
--- a/setup.py
+++ b/setup.py
@@ -95,6 +95,7 @@ def main():
                 "sphinx >= 1.8.0, < 2",
                 "towncrier >= 18.5.0",
                 "pygments-github-lexers >= 0.0.5",
+                "sphinxcontrib-autoprogram >= 0.1.5",
             ],
         },
         classifiers=[
diff --git a/src/tox/cli.py b/src/tox/cli.py
new file mode 100644
index 00000000..2fe755cf
--- /dev/null
+++ b/src/tox/cli.py
@@ -0,0 +1,11 @@
+from tox.config import Parser, get_plugin_manager
+
+
+def cli_parser():
+    parser = Parser()
+    pm = get_plugin_manager(tuple())
+    pm.hook.tox_addoption(parser=parser)
+    return parser.argparser
+
+
+cli = cli_parser()
diff --git a/src/tox/config.py b/src/tox/config.py
index 28173a84..32304299 100755
--- a/src/tox/config.py
+++ b/src/tox/config.py
@@ -343,7 +343,6 @@ class SetenvDict(object):
 
 @tox.hookimpl
 def tox_addoption(parser):
-    # formatter_class=argparse.ArgumentDefaultsHelpFormatter)
     parser.add_argument(
         "--version",
         action="store_true",
diff --git a/tox.ini b/tox.ini
index 3168727e..0548d33e 100644
--- a/tox.ini
+++ b/tox.ini
@@ -34,7 +34,7 @@ commands = pytest {posargs:\
 description = invoke sphinx-build to build the HTML docs
 basepython = python3.7
 extras = docs
-commands = sphinx-build -d "{toxworkdir}/docs_doctree" doc "{toxworkdir}/docs_out" --color -W -bhtml {posargs}
+commands = sphinx-build -d "{toxworkdir}/docs_doctree" docs "{toxworkdir}/docs_out" --color -W -bhtml {posargs}
            python -c 'import pathlib; print("documentation available under file://\{0\}".format(pathlib.Path(r"{toxworkdir}") / "docs_out" / "index.html"))'
 
 [testenv:package_description]


Test patch:
diff --git a/tests/unit/test_docs.py b/tests/unit/test_docs.py
index 4d19e8b3..d62fa784 100644
--- a/tests/unit/test_docs.py
+++ b/tests/unit/test_docs.py
@@ -20,7 +20,7 @@ INI_BLOCK_RE = re.compile(
 
 RST_FILES = []
 TOX_ROOT = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))))
-for root, _, filenames in os.walk(os.path.join(TOX_ROOT, "doc")):
+for root, _, filenames in os.walk(os.path.join(TOX_ROOT, "docs")):
     for filename in filenames:
         if filename.endswith(".rst"):
             RST_FILES.append(os.path.join(root, filename))