Repository: falconry/falcon

Issue:
Adding many routes to CompiledRouter is slow (takes O(n^2) time)
i have about 400 api route to add
falcon compiles method “find” each time and slower as the count grows
it slows down hug booting
does anyone else have same problem？
i made a patch to fix it，but not very elegent，
i wish there will be a better solution

<sub>Sent with <a href="http://githawk.com">GitHawk</a></sub>


Discussion:
Hi @fengyong !
And thanks for reporting the issue.
Indeed, the default [`CompiledRouter`](https://falcon.readthedocs.io/en/stable/api/routing.html?highlight=compiledrouter#id2) is recompiling all the routes upon adding a new one, and as such it has at least **O**(n^2) complexity.

In my test, adding 400 routes takes about 4 seconds, which could probably be still acceptable for some cases? OTOH I realize that may render unit tests unusable slow, as well as one would eventually hit the limit of what is reasonable with even more routes.

Potential solutions:
* Register/inspect the amount of routes already added, and compile in a check to defer compilation to the first routing in case the number of routes is larger than threshold. Recompiling would remove this check, so it would have no implications on performance. Care would need to be taken of handling the first request in a thread-safe manner.
* Allow adding more routes at once, `add_routes`, or similar
i fixed this by add a patch, it works great, but ugly, i wish there will be a better official version
1 preserve origin falcon_compile and set_error_serializer
2 nullify the compile action.
2  wait until a proper time : after calling 'set_error_serializer' , then original falcon compile:

it only takes 2 lines if directly modify falcon's source code, but it need to deliberate the side-affects.

```
def __do_nothing(self):
    pass

def __reset_falcon_find(self):
    self._find = self._compile()


import falcon

origin_falcon_compile = falcon.routing.compiled.CompiledRouter._compile
origin_set_error_serializer = falcon.api.API.set_error_serializer

#disable _compile to save time
falcon.routing.compiled.CompiledRouter._compile = __do_nothing
falcon.routing.compiled.CompiledRouter.__reset_falcon_find = __reset_falcon_find


def wrapper_set_error_serializer(self, serializer):
    falcon.routing.compiled.CompiledRouter._compile = origin_falcon_compile
    self._router.__reset_falcon_find()
    return origin_set_error_serializer(self, serializer)


falcon.api.API.set_error_serializer = wrapper_set_error_serializer

```
4 seconds for 400 routes is acceptable in  fact. 
It just weaken the reputation of python or hug or falcon.  somebody who is not familar with python/falcon will say " python/hug is so slow, we should consider migrate to other languages/framework, like golang".  
this kind of misunderstanding really happened a lot.


> Hi @fengyong !
> And thanks for reporting the issue.
> Indeed, the default [`CompiledRouter`](https://falcon.readthedocs.io/en/stable/api/routing.html?highlight=compiledrouter#id2) is recompiling all the routes upon adding a new one, and as such it has at least **O**(n^2) complexity.
> 
> In my test, adding 400 routes takes about 4 seconds, which could probably be still acceptable for some cases? OTOH I realize that may render unit tests unusable slow, as well as one would eventually hit the limit of what is reasonable with even more routes.
> 
> Potential solutions:
> 
> * Register/inspect the amount of routes already added, and compile in a check to defer compilation to the first routing in case the number of routes is larger than threshold. Recompiling would remove this check, so it would have no implications on performance. Care would need to be taken of handling the first request in a thread-safe manner.
> * Allow adding more routes at once, `add_routes`, or similar


I think it would be reasonable to set a flag that delays code generation until the first call to `find()`. It seems unlikely that a large number of routes would be added after the first attempt to match a route, except perhaps once in a while in a testing context.
As an alternative the every time a new route is added to the router `self._find` could be set to
```py
def _compile_and_find(self, *args):
    self._find = self._compile(...)
    return self._find(*args)
```
so that it will compile only on the first call. This would also avoid the condition to check if a compilation is need in the `find` method

Resolving patch:
diff --git a/falcon/app.py b/falcon/app.py
--- a/falcon/app.py
+++ b/falcon/app.py
@@ -470,6 +470,10 @@ class can use suffixed responders to distinguish requests
                 Another class might use a suffixed responder to handle
                 a shortlink route in addition to the regular route for the
                 resource.
+            compile (bool): Optional flag that can be provided when using the default
+                :class:`.CompiledRouter` to compile the routing logic on this call,
+                since it will otherwise delay compilation until the first request
+                is routed. See :meth:`.CompiledRouter.add_route` for further details.
 
         Note:
             Any additional keyword arguments not defined above are passed
diff --git a/falcon/routing/compiled.py b/falcon/routing/compiled.py
--- a/falcon/routing/compiled.py
+++ b/falcon/routing/compiled.py
@@ -19,6 +19,7 @@
 import keyword
 import re
 import textwrap
+from threading import Lock
 
 from falcon.routing import converters
 from falcon.routing.util import map_http_methods, set_default_responders
@@ -51,6 +52,21 @@ class CompiledRouter:
     tree for each look-up, it generates inlined, bespoke Python code to
     perform the search, then compiles that code. This makes the route
     processing quite fast.
+
+    The compilation process is delayed until the first use of the router (on the
+    first routed request) to reduce the time it takes to start the application.
+    This may noticeably delay the first response of the application when a large
+    number of routes have been added. When adding the last route
+    to the application a `compile` flag may be provided to force the router
+    to compile immediately, thus avoiding any delay for the first response.
+
+    Note:
+        When using a multi-threaded web server to host the application, it is
+        possible that multiple requests may be routed at the same time upon
+        startup. Therefore, the framework employs a lock to ensure that only a
+        single compilation of the decision tree is performed.
+
+    See also :meth:`.CompiledRouter.add_route`
     """
 
     __slots__ = (
@@ -63,6 +79,7 @@ class CompiledRouter:
         '_patterns',
         '_return_values',
         '_roots',
+        '_compile_lock',
     )
 
     def __init__(self):
@@ -80,9 +97,10 @@ def __init__(self):
         self._return_values = None
         self._roots = []
 
-        # NOTE(kgriffs): Call _compile() last since it depends on
-        # the variables above.
-        self._find = self._compile()
+        # NOTE(caselit): set _find to the delayed compile method to ensure that
+        # compile is called when the router is first used
+        self._find = self._compile_and_find
+        self._compile_lock = Lock()
 
     @property
     def options(self):
@@ -90,6 +108,10 @@ def options(self):
 
     @property
     def finder_src(self):
+        # NOTE(caselit): ensure that the router is actually compiled before
+        # returning the finder source, since the current value may be out of
+        # date
+        self.find('/')
         return self._finder_src
 
     def map_http_methods(self, resource, **kwargs):
@@ -142,6 +164,20 @@ class can use suffixed responders to distinguish requests
                 Another class might use a suffixed responder to handle
                 a shortlink route in addition to the regular route for the
                 resource.
+            compile (bool): Optional flag that can be used to compile the
+                routing logic on this call. By default, :class:`.CompiledRouter`
+                delays compilation until the first request is routed. This may
+                introduce a noticeable amount of latency when handling the first
+                request, especially when the application implements a large
+                number of routes. Setting `compile` to ``True`` when the last
+                route is added ensures that the first request will not be
+                delayed in this case (defaults to ``False``).
+
+                Note:
+                    Always setting this flag to ``True`` may slow down the
+                    addition of new routes when hundreds of them are added at
+                    once. It is advisable to only set this flag to ``True`` when
+                    adding the final route.
         """
 
         # NOTE(kgriffs): falcon.asgi.App injects this private kwarg; it is
@@ -206,7 +242,12 @@ def insert(nodes, path_index=0):
                 insert(new_node.children, path_index + 1)
 
         insert(self._roots)
-        self._find = self._compile()
+        # NOTE(caselit): when compile is True run the actual compile step, otherwise reset the
+        # _find, so that _compile will be called on the next find use
+        if kwargs.get('compile', False):
+            self._find = self._compile()
+        else:
+            self._find = self._compile_and_find
 
     def find(self, uri, req=None):
         """Search for a route that matches the given partial URI.
@@ -317,9 +358,15 @@ def _validate_template_segment(self, segment, used_names):
                 raise ValueError(msg)
 
             name = field.group('cname')
-            if name and name not in self._converter_map:
-                msg = 'Unknown converter: "{0}"'.format(name)
-                raise ValueError(msg)
+            if name:
+                if name not in self._converter_map:
+                    msg = 'Unknown converter: "{0}"'.format(name)
+                    raise ValueError(msg)
+                try:
+                    self._instantiate_converter(self._converter_map[name], field.group('argstr'))
+                except Exception as e:
+                    msg = 'Cannot instantiate converter "{}"'.format(name)
+                    raise ValueError(msg) from e
 
     def _generate_ast(self, nodes, parent, return_values, patterns, level=0, fast_return=True):
         """Generates a coarse AST for the router."""
@@ -531,6 +578,25 @@ def _instantiate_converter(self, klass, argstr=None):
         src = '{0}({1})'.format(klass.__name__, argstr)
         return eval(src, {klass.__name__: klass})
 
+    def _compile_and_find(self, path, _return_values, _patterns, _converters, params):
+        """Compile the router, sets the `_find` attribute and returns its result.
+
+        This method is set to the `_find` attribute to delay the compilation of the
+        router until it's used for the first time. Subsequent calls to `_find` will
+        be processed by the actual routing function.
+        This method must have the same signature as the function returned by the
+        :meth:`.CompiledRouter._compile`.
+        """
+        with self._compile_lock:
+            if self._find == self._compile_and_find:
+                # NOTE(caselit): replace the find with the result of the router compilation
+                self._find = self._compile()
+        # NOTE(caselit): return_values, patterns, converters are reset by the _compile
+        # method, so the updated ones must be used
+        return self._find(
+            path, self._return_values, self._patterns, self._converters, params
+        )
+
 
 class CompiledRouterNode:
     """Represents a single URI segment in a URI."""


Test patch:
diff --git a/tests/test_compiled_router.py b/tests/test_compiled_router.py
new file mode 100644
--- /dev/null
+++ b/tests/test_compiled_router.py
@@ -0,0 +1,116 @@
+from threading import Barrier, Thread
+from time import sleep
+from unittest.mock import MagicMock
+
+import pytest
+
+from falcon.routing import CompiledRouter
+
+
+def test_find_src(monkeypatch):
+    called = False
+    find = CompiledRouter.find
+
+    def mock(*args):
+        nonlocal called
+        called = True
+        find(*args)
+
+    monkeypatch.setattr(CompiledRouter, 'find', mock)
+    router = CompiledRouter()
+
+    assert router.finder_src is not None
+    assert called
+
+
+@pytest.fixture
+def patch_add_route(monkeypatch):
+    add_route = CompiledRouter.add_route
+    mock = MagicMock(side_effect=lambda *a, **k: add_route(router, *a, **k))
+    monkeypatch.setattr(CompiledRouter, 'add_route', mock)
+    router = CompiledRouter()
+    return mock, router
+
+
+@pytest.mark.parametrize(
+    'kwargs',
+    (
+        {},
+        {'compile': False},
+        {'compile': False, 'suffix': 'other'},
+        {'suffix': 'other'},
+    ),
+)
+def test_no_compile_kw(patch_add_route, kwargs):
+    mock, router = patch_add_route
+
+    res = MockResource()
+    router.add_route('/foo', res, **kwargs)
+
+    assert mock.call_count == 1
+    mock.assert_has_calls(((('/foo', res), kwargs), ))
+    assert router._find == router._compile_and_find
+
+
+def test_compile(patch_add_route):
+    mock, router = patch_add_route
+
+    res = MockResource()
+    router.add_route('/foo', res, compile=True)
+    assert mock.call_count == 1
+    assert router._find != router._compile_and_find
+
+
+def test_add_route_after_first_request():
+    router = CompiledRouter()
+
+    router.add_route('/foo', MockResource())
+    assert router.find('/foo') is not None
+    assert router._find != router._compile_and_find
+
+    router.add_route('/bar', MockResource(), suffix='other')
+    assert router._find == router._compile_and_find
+    assert router.find('/bar') is not None
+    assert router._find != router._compile_and_find
+
+
+def test_multithread_compile(monkeypatch):
+    def side_effect():
+        sleep(0.05)
+        return lambda *args: None
+
+    mock = MagicMock(side_effect=side_effect)
+    monkeypatch.setattr(CompiledRouter, '_compile', mock)
+
+    router = CompiledRouter()
+    mr = MockResource()
+
+    router.add_route('/foo', mr)
+
+    calls = 0
+    num_threads = 3
+    barrier = Barrier(num_threads)
+
+    def find():
+        nonlocal calls
+        barrier.wait()
+        assert router.find('/foo') is None
+        calls += 1
+
+    threads = [Thread(target=find) for i in range(num_threads)]
+    for t in threads:
+        t.start()
+
+    for t in threads:
+        t.join()
+
+    assert calls == 3
+    mock.assert_called_once_with()
+
+
+class MockResource:
+    def on_get(self, req, res):
+        pass
+
+    def on_get_other(self, req, res):
+        pass
diff --git a/tests/test_default_router.py b/tests/test_default_router.py
--- a/tests/test_default_router.py
+++ b/tests/test_default_router.py
@@ -461,9 +461,11 @@ def test_converters_with_invalid_options(router, uri_template):
     # NOTE(kgriffs): Sanity-check that errors are properly bubbled up
     # when calling add_route(). Additional checks can be found
     # in test_uri_converters.py
-    with pytest.raises(ValueError):
+    with pytest.raises(ValueError, match='Cannot instantiate converter') as e:
         router.add_route(uri_template, ResourceWithId(1))
 
+    assert e.value.__cause__ is not None
+
 
 @pytest.mark.parametrize('uri_template', [
     '/foo/{bar:}',