Repository: open-telemetry/opentelemetry-python

Issue:
Test for package README syntax errors
Add a test that checks for README syntax errors with [`readme_renderer`](https://pypi.org/project/readme_renderer/).

Such errors prevent us from uploading packages to PyPI. See #444.


Discussion:
I'd like to work on this. I thought I'd include it directly into `scripts/eachdist.py` when ` lint --check-only` is run, i.e. the tox lint step. Would that be the best place for it?

It seems there's already a function to get the relevant directories, so I'd just run `setup.py --long-description` on them and feed that into the readme renderer if it's `.rst`.
If it's expressible with little more than a single execute_args call (i.e. you could also do it manually with ` scripts/eachdist.py exec`), then I'd say that, yes, this is a good place for it (following the existing logic in eachdist.py, it should also be run without ` --check-only` though as that flag is only meant to surpress auto-fixing if it is available for the command).
I do not mean to overcomplicate this, but I ran into some questions regarding how to handle this issue.

The only nice way to get the README (or `long_description`) is to run `setup.py —long-description`. There does not seem to be a good way to get the content type (rst/md), however, or the file name of the README.

I could parse those from the `setup.cfg` ini file, if it existed everywhere. But it does not in:

- `opentelemetry-python/docs/examples/opentelemetry-example-app`
- `opentelemetry-python/opentelemetry-api`
- `opentelemetry-python/opentelemetry-sdk`

Are there conventions so that I can just assume that all long descriptions are in rst format? Or even that these are saved in a `README.rst` file so that I can read them directly from that file and complain if it does not exist? Codifying such a convention would be the easiest solution.

Edit: `testutil` is the only distribution without a `README.rst` at the moment but is not being pushed. (It might even be deleted? #374 5., point 4)

It might also be possible to just use ini files for the above packages, but paths with a `pyproject.toml` file are also [returned as targets](https://github.com/open-telemetry/opentelemetry-python/blob/master/scripts/eachdist.py#L254 ). Is there a possibility that `pyproject.toml` might be used instead of `setup.py` and `setup.cfg` in the future?

Resolving patch:
diff --git a/scripts/check_for_valid_readme.py b/scripts/check_for_valid_readme.py
new file mode 100644
--- /dev/null
+++ b/scripts/check_for_valid_readme.py
@@ -0,0 +1,51 @@
+"""Test script to check given paths for valid README.rst files."""
+import argparse
+import sys
+from pathlib import Path
+
+import readme_renderer.rst
+
+
+def is_valid_rst(path):
+    """Checks if RST can be rendered on PyPI."""
+    with open(path) as readme_file:
+        markup = readme_file.read()
+    return readme_renderer.rst.render(markup) is not None
+
+
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="Checks README.rst file in path for syntax errors."
+    )
+    parser.add_argument(
+        "paths", nargs="+", help="paths containing a README.rst to test"
+    )
+    parser.add_argument("-v", "--verbose", action="store_true")
+    return parser.parse_args()
+
+
+def main():
+    args = parse_args()
+    error = False
+
+    for path in map(Path, args.paths):
+        readme = path / "README.rst"
+        try:
+            if not is_valid_rst(readme):
+                error = True
+                print("FAILED: RST syntax errors in", readme)
+                continue
+        except FileNotFoundError:
+            error = True
+            print("FAILED: README.rst not found in", path)
+            continue
+        if args.verbose:
+            print("PASSED:", readme)
+
+    if error:
+        sys.exit(1)
+    print("All clear.")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/scripts/eachdist.py b/scripts/eachdist.py
--- a/scripts/eachdist.py
+++ b/scripts/eachdist.py
@@ -482,6 +482,12 @@ def lint_args(args):
             args, ("exec", "pylint {}", "--all", "--mode", "lintroots")
         )
     )
+    execute_args(
+        parse_subargs(
+            args,
+            ("exec", "python scripts/check_for_valid_readme.py {}", "--all",),
+        )
+    )
 
 
 def test_args(args):


Test patch: