Repository: pytest-dev/pytest
Pull request: adding towncrier wrapper script so 'tox -e docs' works natively on windows
The problem (natively on a windows system):

Before:

```
C:\workspace\pytest>tox -e docs
c:\python37\lib\site-packages\packaging\requirements.py:57: UserWarning: warn_ungrouped_named_tokens_in_collection: setting results name 'specifier' on And expression collides with '_original_end' on contained expression
  VERSION_SPEC = originalTextFor(_VERSION_SPEC)("specifier")
c:\python37\lib\site-packages\packaging\requirements.py:60: UserWarning: warn_ungrouped_named_tokens_in_collection: setting results name 'marker' on And expression collides with '_original_end' on contained expression
  MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")
docs develop-inst-noop: C:\workspace\pytest
docs installed: alabaster==0.7.12,argcomplete==1.11.1,atomicwrites==1.4.0,attrs==19.3.0,Babel==2.8.0,certifi==2020.4.5.1,chardet==3.0.4,click==7.1.2,colorama==0.4.3,docutils==0.16,elementpath==1.4.5,hypothesis==5.15.1,idna==2.9,imag
esize==1.2.0,importlib-metadata==1.6.0,incremental==17.5.0,Jinja2==2.11.2,MarkupSafe==1.1.1,mock==4.0.2,more-itertools==8.3.0,nose==1.3.7,packaging==20.4,pluggy==0.13.1,py==1.8.1,Pygments==2.6.1,pygments-pytest==2.0.0,pyparsing==2.4
.7,-e git+git@github.com:symonk/pytest.git@4a1557fa0e04132eb847d4a5d01c77f0ab777c5a#egg=pytest,pytz==2020.1,requests==2.23.0,six==1.15.0,snowballstemmer==2.0.0,sortedcontainers==2.1.0,Sphinx==2.0.1,sphinx-removed-in==0.2.1,sphinxcon
trib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==1.0.3,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.4,sphinxcontrib-trio==1.1.2,toml==0.10.1,towncrier==19.2.0,u
rllib3==1.25.9,wcwidth==0.1.9,xmlschema==1.1.3,zipp==3.1.0
docs run-test-pre: PYTHONHASHSEED='156'
docs run-test: commands[0] | sh -c 'towncrier --draft > doc/en/_changelog_towncrier_draft.rst'
ERROR: InvocationError for command could not find executable sh
_______________________________________________________________________________________________________________ summary _______________________________________________________________________________________________________________
ERROR:   docs: commands failed
```

After:

```
C:\workspace\pytest>tox -e docs
c:\python37\lib\site-packages\packaging\requirements.py:57: UserWarning: warn_ungrouped_named_tokens_in_collection: setting results name 'specifier' on And expression collides with '_original_end' on contained expression
  VERSION_SPEC = originalTextFor(_VERSION_SPEC)("specifier")
c:\python37\lib\site-packages\packaging\requirements.py:60: UserWarning: warn_ungrouped_named_tokens_in_collection: setting results name 'marker' on And expression collides with '_original_end' on contained expression
  MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")
docs develop-inst-noop: C:\workspace\pytest
docs installed: alabaster==0.7.12,argcomplete==1.11.1,atomicwrites==1.4.0,attrs==19.3.0,Babel==2.8.0,certifi==2020.4.5.1,chardet==3.0.4,click==7.1.2,colorama==0.4.3,docutils==0.16,elementpath==1.4.5,hypothesis==5.15.1,idna==2.9,imag
esize==1.2.0,importlib-metadata==1.6.0,incremental==17.5.0,Jinja2==2.11.2,MarkupSafe==1.1.1,mock==4.0.2,more-itertools==8.3.0,nose==1.3.7,packaging==20.4,pluggy==0.13.1,py==1.8.1,Pygments==2.6.1,pygments-pytest==2.0.0,pyparsing==2.4
.7,-e git+git@github.com:symonk/pytest.git@2b640ff6bee8c3c39e35e1941e7480dae1401848#egg=pytest,pytz==2020.1,requests==2.23.0,six==1.15.0,snowballstemmer==2.0.0,sortedcontainers==2.1.0,Sphinx==2.0.1,sphinx-removed-in==0.2.1,sphinxcon
trib-applehelp==1.0.2,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==1.0.3,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.4,sphinxcontrib-trio==1.1.2,toml==0.10.1,towncrier==19.2.0,u
rllib3==1.25.9,wcwidth==0.1.9,xmlschema==1.1.3,zipp==3.1.0
docs run-test-pre: PYTHONHASHSEED='747'
docs run-test: commands[0] | python scripts/platform_agnostic_towncrier.py
Loading template...
Finding news fragments...
Rendering news fragments...
Draft only -- nothing has been written.
What is seen below is what would be written.

docs run-test: commands[1] | sphinx-build -W --keep-going -b html doc/en doc/en/_build/html -t changelog_towncrier_draft
Running Sphinx v2.0.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [] 0 added, 2 changed, 0 removed
c:\workspace\pytest\.tox\docs\lib\site-packages\sphinx\util\nodes.py:149: FutureWarning:
   The iterable returned by Node.traverse()
   will become an iterator instead of a list in Docutils > 0.16.
  for classifier in reversed(node.parent.traverse(nodes.classifier)):

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] contents
generating indices... genindex
highlighting module code... [100%] pytest
writing additional pages... search
copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded.

The HTML pages are in doc\en\_build\html.
_______________________________________________________________________________________________________________ summary _______________________________________________________________________________________________________________
  docs: commands succeeded
  congratulations :)

C:\workspace\pytest>

```

template populated and created in `doc/en/_changelog_towncrier_draft.rst`


linux/darwin should work too but I'd need to explicitly test it, will get to that shortly.  closes #7251

```
sy@sy-PC MINGW64 /c/workspace/pytest (support-tox-e-docs-on-windows)
$ ls -l doc/en/ | grep change
-rw-r--r-- 1 sy 197121   9873 May 26 16:32 _changelog_towncrier_draft.rst
-rw-r--r-- 1 sy 197121 337393 May 26 13:55 changelog.rst
```




Problem statement:
tox -e docs not supported out of the box on windows platform
When building docs, we use 'sh' which fails from the pycharm terminal on windows for example.  Will investigate if tox can have a platform agnostic command for towncrier that will work for everybody.

```
[testenv:docs]
basepython = python3
usedevelop = True
deps =
    -r{toxinidir}/doc/en/requirements.txt
    towncrier
whitelist_externals = sh
commands =
    sh -c 'towncrier --draft > doc/en/_changelog_towncrier_draft.rst'
    # the '-t changelog_towncrier_draft' tags makes sphinx include the draft
    # changelog in the docs; this does not happen on ReadTheDocs because it uses
    # the standard sphinx command so the 'changelog_towncrier_draft' is never set there
    sphinx-build -W --keep-going -b html doc/en doc/en/_build/html -t changelog_towncrier_draft {posargs:}
```

Hints from issue discussion:
@nicoddemus how does someone get the ability to assign 'labels' on issue(s) on the project here at pytest?
@symonk we would either need a wrapper script around towncrier, or a towncrier feature addition to write the draft to a file
on my radar for this week

Commit message: enable tox -e docs natively on windows using a wrapper

Patch to review:
diff --git a/scripts/platform_agnostic_towncrier.py b/scripts/platform_agnostic_towncrier.py
new file mode 100644
index 00000000000..186d2c4f224
--- /dev/null
+++ b/scripts/platform_agnostic_towncrier.py
@@ -0,0 +1,13 @@
+from subprocess import check_call
+
+
+def main():
+    """
+    Platform agnostic towncrier to support tox -e docs on a windows based platform natively.
+    Previously we used 'sh -c' which will not work on pycharm terminal in windows for example.
+    """
+    check_call("towncrier --draft > doc/en/_changelog_towncrier_draft.rst", shell=True)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/tox.ini b/tox.ini
index f363f57012e..052e8658184 100644
--- a/tox.ini
+++ b/tox.ini
@@ -80,9 +80,8 @@ usedevelop = True
 deps =
     -r{toxinidir}/doc/en/requirements.txt
     towncrier
-whitelist_externals = sh
 commands =
-    sh -c 'towncrier --draft > doc/en/_changelog_towncrier_draft.rst'
+    python scripts/platform_agnostic_towncrier.py
     # the '-t changelog_towncrier_draft' tags makes sphinx include the draft
     # changelog in the docs; this does not happen on ReadTheDocs because it uses
     # the standard sphinx command so the 'changelog_towncrier_draft' is never set there


Code review comments:
diff_hunk: @@ -0,0 +1,13 @@
+from subprocess import check_call
+
+
+def main():
+    """
+    Platform agnostic towncrier to support tox -e docs on a windows based platform natively.
+    Previously we used 'sh -c' which will not work on pycharm terminal in windows for example.
+    """
+    check_call("towncrier --draft > doc/en/_changelog_towncrier_draft.rst", shell=True)
path: scripts/platform_agnostic_towncrier.py
text: @user1:
shell=True is generally a security problem -- you can do this in pure python with:

```python
with open(..., 'w') as f:
    subprocess.check_call(('towncrier', '--draft'), stdout=f)
```