Repository: pypa/readme_renderer (language: python)

Issue:
bibtex entry on PyPI/TestPyPI renders strikethrough element 
 around curly braces
<!--
    NOTE: This issue should be for problems with PyPI itself, including:
    * pypi.org
    * test.pypi.org
    * files.pythonhosted.org

    This issue should NOT be for a project installed from PyPI. If you are
    having an issue with a specific package, you should reach out to the
    maintainers of that project directly instead.

    Furthermore, this issue should NOT be for any non-PyPI properties (like
    python.org, docs.python.org, etc.)

    If your problem is related to search (a new or updated project doesn't
    appear in the PyPI search results), please wait for a couple of hours
    and check again before reporting it. The search index may take some
    time to be updated.
-->

**Describe the bug**
<!-- A clear and concise description the bug -->

This is https://github.com/scikit-hep/pyhf/issues/2024.

On PyPI and TestPyPI the following RST

```rst
.. code:: bibtex

   @software{pyhf,
     author = {Lukas Heinrich and Matthew Feickert and Giordon Stark},
     title = "{pyhf: v0.7.0rc4}",
     version = {0.7.0rc4},
     doi = {10.5281/zenodo.1169739},
     url = {https://doi.org/10.5281/zenodo.1169739},
     note = {https://github.com/scikit-hep/pyhf/releases/tag/v0.7.0rc4}
   }

   @article{pyhf_joss,
     doi = {10.21105/joss.02823},
     url = {https://doi.org/10.21105/joss.02823},
     year = {2021},
     publisher = {The Open Journal},
     volume = {6},
     number = {58},
     pages = {2823},
     author = {Lukas Heinrich and Matthew Feickert and Giordon Stark and Kyle Cranmer},
     title = {pyhf: pure-Python implementation of HistFactory statistical models},
     journal = {Journal of Open Source Software}
   }
```

gets rendered out with the strikethrough element `
` around curly braces:

```
@software{pyhf,
  author = 
{Lukas Heinrich and Matthew Feickert and Giordon Stark}
,
  title = 
"{pyhf: v0.7.0rc4}"
,
  version = 
{0.7.0rc4}
,
  doi = 
{10.5281/zenodo.1169739}
,
  url = 
{https://doi.org/10.5281/zenodo.1169739}
,
  note = 
{https://github.com/scikit-hep/pyhf/releases/tag/v0.7.0rc4}

}

@article{pyhf_joss,
  doi = 
{10.21105/joss.02823}
,
  url = 
{https://doi.org/10.21105/joss.02823}
,
  year = 
{2021}
,
  publisher = 
{The Open Journal}
,
  volume = 
{6}
,
  number = 
{58}
,
  pages = 
{2823}
,
  author = 
{Lukas Heinrich and Matthew Feickert and Giordon Stark and Kyle Cranmer}
,
  title = 
{pyhf: pure-Python implementation of HistFactory statistical models}
,
  journal = 
{Journal of Open Source Software}

}
```

**Expected behavior**
<!-- A clear and concise description of what you expected to happen -->

I would expect that it get rendered out correctly without the `
` like it is in the [project GitHub README](https://github.com/scikit-hep/pyhf/blob/c2710e9b476e4eb9a1d6ee4df1accc4e024fc635/README.rst#citation) and on the [project docs](https://pyhf.readthedocs.io/en/v0.6.3/#citation).

**To Reproduce**
<!-- Steps to reproduce the bug, or a link to PyPI where the bug is visible -->

* On PyPI: https://pypi.org/project/pyhf/0.6.3/
* On TestPyPI: https://test.pypi.org/project/pyhf/0.7.0rc4/

**My Platform**
<!--
    Any details about your specific platform:
    * If the problem is in the browser, what browser, version, and OS?
    * If the problem is with a command-line tool, what version of that tool?
    * If the problem is with connecting to PyPI, include some details about
      your network, including SSL/TLS implementation in use, internet service
      provider, and if there are any firewalls or proxies in use.
-->

This is not platform dependent. I've tested this on Chrome and Firefox on both Linux and Mac.

**Additional context**
<!-- Add any other context, links, etc. about the feature here. -->

None.


Discussion / hints:
Thanks for the detailed example, I'll take a deeper look at this soon. 
This seems to stem from the setting in place to support Pygments highlighting:
https://github.com/pypa/readme_renderer/blob/e662bf491306d312ddf3ed1ea93355360013b845/readme_renderer/rst.py#L96-L98

Removing this setting extracts the expected content. We use the Pygments short codes in warehouse CSS here: https://github.com/pypi/warehouse/blob/665eb50ce0066afa555f34c1ae5637bdd9361b59/warehouse/static/sass/blocks/_project-description.scss#L283-L516

It's unclear why docutils would behave differently for bibtex for this section only.
Digging further, it's not docutils at all, it's driven from Pygments syntax highlighter, as it establishes a short name token for syntax classes, and has been `s` for years. https://github.com/pygments/pygments/blob/b58e855e41b9266f8412823dbb260d1b8e49b45f/pygments/token.py#L165

It looks like with the changeover to the HTML5 writer in #253 the `
` attribute wasn't allowed through our cleaner, and is thus escaped to a string value of `&lt;s&gt;{2021}&lt;/s&gt;` instead of `
{2021}
` which [warehouse CSS](https://github.com/pypi/warehouse/blob/665eb50ce0066afa555f34c1ae5637bdd9361b59/warehouse/static/sass/blocks/_project-description.scss#L376-L378) would pick up and handle as `.project-description.s` appropriately.

Fix patch:
diff --git a/readme_renderer/clean.py b/readme_renderer/clean.py
index 747bab8..386e684 100644
--- a/readme_renderer/clean.py
+++ b/readme_renderer/clean.py
@@ -30,7 +30,7 @@
     "br", "caption", "cite", "col", "colgroup", "dd", "del", "details", "div",
     "dl", "dt", "h1", "h2", "h3", "h4", "h5", "h6", "hr", "img", "p", "pre",
     "span", "sub", "summary", "sup", "table", "tbody", "td", "th", "thead",
-    "tr", "tt", "kbd", "var", "input", "section", "aside", "nav",
+    "tr", "tt", "kbd", "var", "input", "section", "aside", "nav", "s",
 ]
 
 ALLOWED_ATTRIBUTES = {


Test patch:
diff --git a/tests/fixtures/test_rst_bibtex.html b/tests/fixtures/test_rst_bibtex.html
new file mode 100644
index 0000000..4305801
--- /dev/null
+++ b/tests/fixtures/test_rst_bibtex.html
@@ -0,0 +1,2 @@
+<pre><code><span class="nc">&#64;article</span><span class="p">{</span><span class="nl">the_impact_of_pygments_docutils_config_and_html5</span><span class="p">,</span><span class="w">
+  </span><span class="na">year</span><span class="w"> </span><span class="p">=</span><span class="w"> </span>
{2022}
<span class="p">,</span></code></pre>
diff --git a/tests/fixtures/test_rst_bibtex.rst b/tests/fixtures/test_rst_bibtex.rst
new file mode 100644
index 0000000..a7642f4
--- /dev/null
+++ b/tests/fixtures/test_rst_bibtex.rst
@@ -0,0 +1,4 @@
+.. code:: bibtex
+
+   @article{the_impact_of_pygments_docutils_config_and_html5,
+     year = {2022},