Repository: pypi/warehouse

Issue:
XML-RPC changelog_since_serial() method is returning invalid XML due to \x01 in sdist name
As of 2019-04-01 17:35:00 +0000, running the below Python 3 code:

```
#!/usr/bin/python3
import csv
from   xmlrpc.client import ServerProxy
import sys

ENDPOINT = 'https://pypi.org/pypi'
SINCE = 5011848
   
client = ServerProxy(ENDPOINT, use_builtin_types=True)
out = csv.writer(sys.stdout)
out.writerow('name version timestamp action serial'.split())
for row in client.changelog_since_serial(SINCE):
    out.writerow(row)
```

fails with:

```
Traceback (most recent call last):
  File "minimum.py", line 12, in <module>
    for row in client.changelog_since_serial(SINCE):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 1112, in __call__
    return self.__send(self.__name, args)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 1452, in __request
    verbose=self.__verbose
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 1154, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 1170, in single_request
    return self.parse_response(resp)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 1336, in parse_response
    p.feed(data)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/xmlrpc/client.py", line 439, in feed
    self._parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 5264, column 38
```

Manually reproducing the XML-RPC request and inspecting the response shows that the problematic piece of XML is (with line number added):

```
<value><array><data>
<value><string>request-s</string></value>
<value><string>0.0.1</string></value>
<value><int>1554118234</int></value>
<value><string>add source file request^As-0.0.1.tar.gz</string></value> <!-- line 5364 -->
<value><int>5016503</int></value>
</data></array></value>
```

The `^A` is here a literal `\x01` which somehow ended up in the filename of an sdist uploaded to the `request-s` project.  If Warehouse is going to allow uploads of such filenames, it needs to properly escape them in XML-RPC responses.


Discussion:
Upon further investigation, it appears that the failure to escape `\x01` is the fault of Python's `xmlrpc.client.dumps()`.  BPO 26147 appears to be related.
Here's the project in question: https://pypi.org/project/request-s/

Here's where we check that it normalizes to the same value as the project name: https://github.com/pypa/warehouse/blob/a3036d44179f640be52578d0d07fce75fe4793c5/warehouse/forklift/legacy.py#L1073-L1080

AFAICT, this is a valid, albeit confusing filename, and the issue is with the client instead.
@di is this a place where the addition of some calls to [`_clean_for_xml`](https://github.com/pypa/warehouse/blob/a3036d44179f640be52578d0d07fce75fe4793c5/warehouse/legacy/api/xmlrpc/views.py#L79-L89) are needed?
@ewdurbin Not sure. Cleaning the description/summary is one thing, but actually changing the filenames might not be helpful.
It appears that `\x01` (and most C0 control characters) is flat-out invalid, both in raw and `&#x01;` form, in XML 1.0 (which is the version of XML that PyPI returns and seems to be the only version Python's XML libraries are aware of).  Hence, the problem is with the very attempt to transmit the string "request\x01-s" over XML-RPC.
I think given the relative infrequency of files with these characters in them, and given the "deprecation" of the XML-RPC API, we can probably just use `_clean_for_xml` on the filenames to ensure this isn't breaking mirrors all over the place.
it seems reasonable to at least clean the changelog entries. will PR that shortly.
Though on second thought, I'm fairly certain this will also effect the releases data as well.
seems url encoding the filename field is probably the safer bet.

Resolving patch:
diff --git a/warehouse/legacy/api/xmlrpc/views.py b/warehouse/legacy/api/xmlrpc/views.py
--- a/warehouse/legacy/api/xmlrpc/views.py
+++ b/warehouse/legacy/api/xmlrpc/views.py
@@ -492,7 +492,7 @@ def changelog_since_serial(request, serial: int):
             e.name,
             e.version,
             int(e.submitted_date.replace(tzinfo=datetime.timezone.utc).timestamp()),
-            e.action,
+            _clean_for_xml(e.action),
             e.id,
         )
         for e in entries


Test patch: