Repository: aio-libs/aiohttp

Issue:
aiohttp webserver with python 3.7 broken
I have python3.7-dev and python-nightly in my CI script. Recently they started failing when I call  app.router.add_get()

This is a minimal reproducer, works on 3.6 and fails on 3.7-dev
```
import re

r = re.compile(re.escape('/foo'))
print(r)
print(r.pattern)
assert r.pattern.startswith('\\/')
```

Python 3.7-dev output:
```
re.compile('/foo')
/foo
Traceback (most recent call last):
 File "aiohttp-issue.py", line 6, in <module>
   assert r.pattern.startswith('\\/')
AssertionError
```
python 3.6.0 output (success):
```
re.compile('\\/foo')
\/foo
```
This is the actual traceback my full app gets in 3.7-dev and aiohttp-2.0.7:
```
Traceback (most recent call last):
  File "../scripts/crawl.py", line 89, in <module>
    main()
  File "../scripts/crawl.py", line 67, in main
    app = webserver.make_app(loop, config)
  File "/home/travis/build/cocrawler/cocrawler/cocrawler/webserver.py", line 16, in make_app
    app.router.add_get('/api/{name}', api)
  File "/home/travis/virtualenv/python3.7-dev/lib/python3.7/site-packages/aiohttp/web_urldispatcher.py", line 865, in add_get
    self.add_route(hdrs.METH_HEAD, *args, name=head_name, **kwargs)
  File "/home/travis/virtualenv/python3.7-dev/lib/python3.7/site-packages/aiohttp/web_urldispatcher.py", line 823, in add_rout
    resource = self.add_resource(path, name=name)
  File "/home/travis/virtualenv/python3.7-dev/lib/python3.7/site-packages/aiohttp/web_urldispatcher.py", line 817, in add_resource
    resource = DynamicResource(compiled, formatter, name=name)
  File "/home/travis/virtualenv/python3.7-dev/lib/python3.7/site-packages/aiohttp/web_urldispatcher.py", line 332, in __init__
    assert pattern.pattern.startswith('\\/')
AssertionError
```
## Your environment

The above 3.6 and 3.7 output was run on CentOS 7 with python installed by pyenv.




Discussion:
That's seems like regression in cpython.
Oh, right, I didn't check the documentation for "re" -- after looking at compiled.pattern I think you're correct.
http://bugs.python.org/issue30147

The python folks say that re.escape() was escaping too much in the past, and has been fixed. Their response to my bug report was to add more documentation to "porting to 3.7".

http://bugs.python.org/issue29995 -- bug for re.escape over-escaping

http://bugs.python.org/issue30147 -- my bug report

So, basically, they say your test is now wrong.
does it work with "python -OO "?
Yes, python 3.7-dev passes with "python -OO". Which is pretty weird.

```
$ pyenv local 3.7-dev
$ python bug.py
re.compile('/foo')
/foo
Traceback (most recent call last):
  File "bug.py", line 6, in <module>
    assert r.pattern.startswith('\\/')
AssertionError
$ python -OO bug.py
re.compile('/foo')
/foo
```
but does web server work?
@asvetlov could you look into this problem too?

Resolving patch:
diff --git a/aiohttp/web_urldispatcher.py b/aiohttp/web_urldispatcher.py
--- a/aiohttp/web_urldispatcher.py
+++ b/aiohttp/web_urldispatcher.py
@@ -30,6 +30,7 @@
            'StaticResource', 'View')
 
 HTTP_METHOD_RE = re.compile(r"^[0-9A-Za-z!#\$%&'\*\+\-\.\^_`\|~]+$")
+PATH_SEP = re.escape('/')
 
 
 class AbstractResource(Sized, Iterable):
@@ -329,7 +330,7 @@ class DynamicResource(Resource):
 
     def __init__(self, pattern, formatter, *, name=None):
         super().__init__(name=name)
-        assert pattern.pattern.startswith('\\/')
+        assert pattern.pattern.startswith(PATH_SEP)
         assert formatter.startswith('/')
         self._pattern = pattern
         self._formatter = formatter


Test patch:
diff --git a/tests/test_urldispatch.py b/tests/test_urldispatch.py
--- a/tests/test_urldispatch.py
+++ b/tests/test_urldispatch.py
@@ -12,8 +12,8 @@
 from aiohttp import hdrs, web
 from aiohttp.test_utils import make_mocked_request
 from aiohttp.web import HTTPMethodNotAllowed, HTTPNotFound, Response
-from aiohttp.web_urldispatcher import (AbstractResource, ResourceRoute,
-                                       SystemRoute, View,
+from aiohttp.web_urldispatcher import (PATH_SEP, AbstractResource,
+                                       ResourceRoute, SystemRoute, View,
                                        _defaultExpectHandler)
 
 
@@ -495,8 +495,11 @@ def test_add_route_with_invalid_re(router):
     with pytest.raises(ValueError) as ctx:
         router.add_route('GET', r'/handler/{to:+++}', handler)
     s = str(ctx.value)
-    assert s.startswith(
-        "Bad pattern '\/handler\/(?P<to>+++)': nothing to repeat")
+    assert s.startswith("Bad pattern '" +
+                        PATH_SEP +
+                        "handler" +
+                        PATH_SEP +
+                        "(?P<to>+++)': nothing to repeat")
     assert ctx.value.__cause__ is None
 
 
@@ -798,7 +801,7 @@ def test_match_info_get_info_dynamic(router):
     req = make_request('GET', '/value')
     info = yield from router.resolve(req)
     assert info.get_info() == {
-        'pattern': re.compile('\\/(?P<a>[^{}/]+)'),
+        'pattern': re.compile(PATH_SEP+'(?P<a>[^{}/]+)'),
         'formatter': '/{a}'}
 
 
@@ -809,7 +812,10 @@ def test_match_info_get_info_dynamic2(router):
     req = make_request('GET', '/path/to')
     info = yield from router.resolve(req)
     assert info.get_info() == {
-        'pattern': re.compile('\\/(?P<a>[^{}/]+)\\/(?P<b>[^{}/]+)'),
+        'pattern': re.compile(PATH_SEP +
+                              '(?P<a>[^{}/]+)' +
+                              PATH_SEP +
+                              '(?P<b>[^{}/]+)'),
         'formatter': '/{a}/{b}'}