Repository: falconry/falcon

Issue:
Route to static file
On a PaaS you may not be able to server static files from the web server directly, so let's make it easy to do this in Falcon.

Depends on #181 


Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=h1) Report
> Merging [#1157](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=desc) into [master](https://codecov.io/gh/falconry/falcon/commit/064232a826070c8ff528b739e560917e7da67e1e?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/falconry/falcon/pull/1157/graphs/tree.svg?width=650&token=zOtkfNQ3h6&height=150&src=pr)](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=tree)

```diff
@@          Coverage Diff           @@
##           master   #1157   +/-   ##
======================================
  Coverage     100%    100%           
======================================
  Files          36      37    +1     
  Lines        2378    2421   +43     
  Branches      347     353    +6     
======================================
+ Hits         2378    2421   +43
```


| [Impacted Files](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [falcon/response\_helpers.py](https://codecov.io/gh/falconry/falcon/pull/1157/diff?src=pr&el=tree#diff-ZmFsY29uL3Jlc3BvbnNlX2hlbHBlcnMucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [falcon/routing/static.py](https://codecov.io/gh/falconry/falcon/pull/1157/diff?src=pr&el=tree#diff-ZmFsY29uL3JvdXRpbmcvc3RhdGljLnB5) | `100% <100%> (ø)` | |
| [falcon/routing/\_\_init\_\_.py](https://codecov.io/gh/falconry/falcon/pull/1157/diff?src=pr&el=tree#diff-ZmFsY29uL3JvdXRpbmcvX19pbml0X18ucHk=) | `100% <100%> (ø)` | :arrow_up: |
| [falcon/response.py](https://codecov.io/gh/falconry/falcon/pull/1157/diff?src=pr&el=tree#diff-ZmFsY29uL3Jlc3BvbnNlLnB5) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=footer). Last update [064232a...869d993](https://codecov.io/gh/falconry/falcon/pull/1157?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Patch:
diff --git a/falcon/api.py b/falcon/api.py
index beb4fff..a003f6d 100644
--- a/falcon/api.py
+++ b/falcon/api.py
@@ -140,7 +140,8 @@ class API(object):
     __slots__ = ('_request_type', '_response_type',
                  '_error_handlers', '_media_type', '_router', '_sinks',
                  '_serialize_error', 'req_options', 'resp_options',
-                 '_middleware', '_independent_middleware', '_router_search')
+                 '_middleware', '_independent_middleware', '_router_search',
+                 '_static_routes')
 
     def __init__(self, media_type=DEFAULT_MEDIA_TYPE,
                  request_type=Request, response_type=Response,
@@ -148,6 +149,7 @@ class API(object):
                  independent_middleware=False):
         self._sinks = []
         self._media_type = media_type
+        self._static_routes = []
 
         # set middleware
         self._middleware = helpers.prepare_middleware(
@@ -350,6 +352,53 @@ class API(object):
         self._router.add_route(uri_template, method_map, resource, *args,
                                **kwargs)
 
+    def add_static_route(self, prefix, directory, downloadable=False):
+        """Add a route to a directory of static files.
+
+        Static routes provide a way to serve files directly. This
+        feature provides an alternative to serving files at the web server
+        level when you don't have that option, when authorization is
+        required, or for testing purposes.
+
+        Warning:
+            Serving files directly from the web server,
+            rather than through the Python app, will always be more efficient,
+            and therefore should be preferred in production deployments.
+
+        Static routes are matched in LIFO order. Therefore, if the same
+        prefix is used for two routes, the second one will override the
+        first. This also means that more specific routes should be added
+        *after* less specific ones. For example, the following sequence
+        would result in ``'/foo/bar/thing.js'`` being mapped to the
+        ``'/foo/bar'`` route, and ``'/foo/xyz/thing.js'`` being mapped to the
+        ``'/foo'`` route::
+
+            api.add_static_route('/foo', foo_path)
+            api.add_static_route('/foo/bar', foobar_path)
+
+        Args:
+            prefix (str): The path prefix to match for this route. If the
+                path in the requested URI starts with this string, the remainder
+                of the path will be appended to the source directory to
+                determine the file to serve. This is done in a secure manner
+                to prevent an attacker from requesting a file outside the
+                specified directory.
+
+                Note that static routes are matched in LIFO order, and are only
+                attempted after checking dynamic routes and sinks.
+
+            directory (str): The source directory from which to serve files.
+            downloadable (bool): Set to ``True`` to include a
+                Content-Disposition header in the response. The "filename"
+                directive is simply set to the name of the requested file.
+
+        """
+
+        self._static_routes.insert(
+            0,
+            routing.StaticRoute(prefix, directory, downloadable=downloadable)
+        )
+
     def add_sink(self, sink, prefix=r'/'):
         """Register a sink method for the API.
 
@@ -563,7 +612,13 @@ class API(object):
 
                     break
             else:
-                responder = falcon.responders.path_not_found
+
+                for sr in self._static_routes:
+                    if sr.match(path):
+                        responder = sr
+                        break
+                else:
+                    responder = falcon.responders.path_not_found
 
         return (responder, params, resource, uri_template)
 
diff --git a/falcon/response.py b/falcon/response.py
index e10d778..9479468 100644
--- a/falcon/response.py
+++ b/falcon/response.py
@@ -14,6 +14,8 @@
 
 """Response class."""
 
+import mimetypes
+
 from six import PY2
 from six import string_types as STRING_TYPES
 
@@ -25,6 +27,7 @@ from six.moves import http_cookies  # NOQA: I202
 from falcon import DEFAULT_MEDIA_TYPE
 from falcon.media import Handlers
 from falcon.response_helpers import (
+    format_content_disposition,
     format_header_value_list,
     format_range,
     header_property,
@@ -34,6 +37,7 @@ from falcon.util import dt_to_http, TimezoneGMT
 from falcon.util.uri import encode as uri_encode
 from falcon.util.uri import encode_value as uri_encode_value
 
+
 SimpleCookie = http_cookies.SimpleCookie
 CookieError = http_cookies.CookieError
 
@@ -679,6 +683,16 @@ class Response(object):
         and ``falcon.MEDIA_GIF``.
         """)
 
+    downloadable_as = header_property(
+        'Content-Disposition',
+        """Set the Content-Disposition header using the given filename.
+
+        The value will be used for the "filename" directive. For example,
+        given 'report.pdf', the Content-Disposition header would be set
+        to ``'attachment; filename="report.pdf"'``.
+        """,
+        format_content_disposition)
+
     etag = header_property(
         'ETag',
         'Set the ETag header.')
@@ -811,24 +825,32 @@ class ResponseOptions(object):
             not requiring HTTPS. Note, however, that this setting can
             be overridden via `set_cookie()`'s `secure` kwarg.
 
-        default_media_type (str): The default media-type to use when
-            deserializing a response. This value is normally set to the media
-            type provided when a :class:`falcon.API` is initialized; however,
-            if created independently, this will default to the
+        default_media_type (str): The default Internet media type (RFC 2046) to
+            use when deserializing a response. This value is normally set to the
+            media type provided when a :class:`falcon.API` is initialized;
+            however, if created independently, this will default to the
             ``DEFAULT_MEDIA_TYPE`` specified by Falcon.
 
         media_handlers (Handlers): A dict-like object that allows you to
             configure the media-types that you would like to handle.
             By default, a handler is provided for the ``application/json``
             media type.
+
+        static_media_types (dict): A mapping of dot-prefixed file extensions to
+            Internet media types (RFC 2046). Defaults to ``mimetypes.types_map``
+            after calling ``mimetypes.init()``.
     """
     __slots__ = (
         'secure_cookies_by_default',
         'default_media_type',
         'media_handlers',
+        'static_media_types',
     )
 
     def __init__(self):
         self.secure_cookies_by_default = True
         self.default_media_type = DEFAULT_MEDIA_TYPE
         self.media_handlers = Handlers()
+
+        mimetypes.init()
+        self.static_media_types = mimetypes.types_map
diff --git a/falcon/response_helpers.py b/falcon/response_helpers.py
index 47308c2..602eb5b 100644
--- a/falcon/response_helpers.py
+++ b/falcon/response_helpers.py
@@ -77,6 +77,12 @@ def format_range(value):
     return result
 
 
+def format_content_disposition(value):
+    """Formats a Content-Disposition header given a filename."""
+
+    return 'attachment; filename="' + value + '"'
+
+
 if six.PY2:
     def format_header_value_list(iterable):
         """Join an iterable of strings with commas."""
diff --git a/falcon/routing/__init__.py b/falcon/routing/__init__.py
index abb9c87..51bc7d3 100644
--- a/falcon/routing/__init__.py
+++ b/falcon/routing/__init__.py
@@ -20,6 +20,7 @@ routers.
 """
 
 from falcon.routing.compiled import CompiledRouter, CompiledRouterOptions  # NOQA
+from falcon.routing.static import StaticRoute  # NOQA
 from falcon.routing.util import create_http_method_map  # NOQA
 from falcon.routing.util import map_http_methods  # NOQA
 from falcon.routing.util import set_default_responders  # NOQA
diff --git a/falcon/routing/static.py b/falcon/routing/static.py
new file mode 100644
index 0000000..ba3fab2
--- /dev/null
+++ b/falcon/routing/static.py
@@ -0,0 +1,98 @@
+import io
+import os
+import re
+
+import falcon
+
+
+class StaticRoute(object):
+    """Represents a static route.
+
+    Args:
+        prefix (str): The path prefix to match for this route. If the
+            path in the requested URI starts with this string, the remainder
+            of the path will be appended to the source directory to
+            determine the file to serve. This is done in a secure manner
+            to prevent an attacker from requesting a file outside the
+            specified directory.
+
+            Note that static routes are matched in LIFO order, and are only
+            attempted after checking dynamic routes and sinks.
+
+        directory (str): The source directory from which to serve files. Must
+            be an absolute path.
+        downloadable (bool): Set to ``True`` to include a
+            Content-Disposition header in the response. The "filename"
+            directive is simply set to the name of the requested file.
+    """
+
+    # NOTE(kgriffs): Don't allow control characters and reserved chars
+    _DISALLOWED_CHARS_PATTERN = re.compile('[\x00-\x1f\x80-\x9f~?<>:*|\'"]')
+
+    # NOTE(kgriffs): If somehow an executable code exploit is triggerable, this
+    # minimizes how much can be included in the payload.
+    _MAX_NON_PREFIXED_LEN = 512
+
+    def __init__(self, prefix, directory, downloadable=False):
+        if not prefix.startswith('/'):
+            raise ValueError("prefix must start with '/'")
+
+        if not os.path.isabs(directory):
+            raise ValueError('directory must be an absolute path')
+
+        # NOTE(kgriffs): Ensure it ends with a path separator to ensure
+        # we only match on the complete segment. Don't raise an error
+        # because most people won't expect to have to append a slash.
+        if not prefix.endswith('/'):
+            prefix += '/'
+
+        self._prefix = prefix
+        self._directory = directory
+        self._downloadable = downloadable
+
+    def match(self, path):
+        """Check whether the given path matches this route."""
+        return path.startswith(self._prefix)
+
+    def __call__(self, req, resp):
+        """Resource responder for this route."""
+
+        without_prefix = req.path[len(self._prefix):]
+
+        # NOTE(kgriffs): Check surrounding whitespace and strip trailing
+        # periods, which are illegal on windows
+        if (not without_prefix or
+                without_prefix.strip().rstrip('.') != without_prefix or
+                self._DISALLOWED_CHARS_PATTERN.search(without_prefix) or
+                '\\' in without_prefix or
+                '//' in without_prefix or
+                len(without_prefix) > self._MAX_NON_PREFIXED_LEN):
+
+            raise falcon.HTTPNotFound()
+
+        normalized = os.path.normpath(without_prefix)
+
+        if normalized.startswith('../') or normalized.startswith('/'):
+            raise falcon.HTTPNotFound()
+
+        file_path = os.path.join(self._directory, normalized)
+
+        # NOTE(kgriffs): Final sanity-check just to be safe. This check
+        # should never succeed, but this should guard against us having
+        # overlooked something.
+        if '..' in file_path or not file_path.startswith(self._directory):
+            raise falcon.HTTPNotFound()  # pragma: nocover
+
+        try:
+            resp.stream = io.open(file_path, 'rb')
+        except IOError:
+            raise falcon.HTTPNotFound()
+
+        suffix = os.path.splitext(file_path)[1]
+        resp.content_type = resp.options.static_media_types.get(
+            suffix,
+            'application/octet-stream'
+        )
+
+        if self._downloadable:
+            resp.downloadable_as = os.path.basename(file_path)


Test patch:
diff --git a/tests/test_headers.py b/tests/test_headers.py
index 0f03ecc..79ab1df 100644
--- a/tests/test_headers.py
+++ b/tests/test_headers.py
@@ -52,6 +52,7 @@ class HeaderHelpersResource(object):
         # Relative URI's are OK per http://goo.gl/DbVqR
         resp.location = '/things/87'
         resp.content_location = '/things/78'
+        resp.downloadable_as = 'Some File.zip'
 
         if req.range_unit is None or req.range_unit == 'bytes':
             # bytes 0-499/10240
@@ -310,6 +311,7 @@ class TestHeaders(object):
         content_type = 'x-falcon/peregrine'
         assert resp.content_type == content_type
         assert result.headers['Content-Type'] == content_type
+        assert result.headers['Content-Disposition'] == 'attachment; filename="Some File.zip"'
 
         cache_control = ('public, private, no-cache, no-store, '
                          'must-revalidate, proxy-revalidate, max-age=3600, '
diff --git a/tests/test_static.py b/tests/test_static.py
new file mode 100644
index 0000000..2fbb55c
--- /dev/null
+++ b/tests/test_static.py
@@ -0,0 +1,201 @@
+# -*- coding: utf-8 -*-
+
+import io
+
+import pytest
+
+import falcon
+from falcon.request import Request
+from falcon.response import Response
+from falcon.routing import StaticRoute
+import falcon.testing as testing
+
+
+@pytest.fixture
+def client():
+    app = falcon.API()
+    return testing.TestClient(app)
+
+
+@pytest.mark.parametrize('uri', [
+    # Root
+    '/static',
+    '/static/',
+    '/static/.',
+
+    # Attempt to jump out of the directory
+    '/static/..',
+    '/static/../.',
+    '/static/.././etc/passwd',
+    '/static/../etc/passwd',
+    '/static/css/../../secret',
+    '/static/css/../../etc/passwd',
+    '/static/./../etc/passwd',
+
+    # The file system probably won't process escapes, but better safe than sorry
+    '/static/css/../.\\056/etc/passwd',
+    '/static/./\\056./etc/passwd',
+    '/static/\\056\\056/etc/passwd',
+
+    # Double slash
+    '/static//test.css',
+    '/static//COM10',
+    '/static/path//test.css',
+    '/static/path///test.css',
+    '/static/path////test.css',
+    '/static/path/foo//test.css',
+
+    # Control characters (0x00–0x1f and 0x80–0x9f)
+    '/static/.\x00ssh/authorized_keys',
+    '/static/.\x1fssh/authorized_keys',
+    '/static/.\x80ssh/authorized_keys',
+    '/static/.\x9fssh/authorized_keys',
+
+    # Reserved characters (~, ?, <, >, :, *, |, ', and ")
+    '/static/~/.ssh/authorized_keys',
+    '/static/.ssh/authorized_key?',
+    '/static/.ssh/authorized_key>foo',
+    '/static/.ssh/authorized_key|foo',
+    '/static/.ssh/authorized_key<foo',
+    '/static/something:something',
+    '/static/thing*.sql',
+    '/static/\'thing\'.sql',
+    '/static/"thing".sql',
+
+    # Trailing periods and spaces
+    '/static/something.',
+    '/static/something..',
+    '/static/something ',
+    '/static/ something ',
+    '/static/ something ',
+    '/static/something\t',
+    '/static/\tsomething',
+
+    # Too long
+    '/static/' + ('t' * StaticRoute._MAX_NON_PREFIXED_LEN) + 'x',
+
+])
+def test_bad_path(uri, monkeypatch):
+    monkeypatch.setattr(io, 'open', lambda path, mode: path)
+
+    sr = StaticRoute('/static', '/var/www/statics')
+
+    req = Request(testing.create_environ(
+        host='test.com',
+        path=uri,
+        app='statics'
+    ))
+
+    resp = Response()
+
+    with pytest.raises(falcon.HTTPNotFound):
+        sr(req, resp)
+
+
+@pytest.mark.parametrize('prefix, directory', [
+    ('static', '/var/www/statics'),
+    ('/static', './var/www/statics'),
+    ('/static', 'statics'),
+    ('/static', '../statics'),
+])
+def test_invalid_args(prefix, directory, client):
+    with pytest.raises(ValueError):
+        StaticRoute(prefix, directory)
+
+    with pytest.raises(ValueError):
+        client.app.add_static_route(prefix, directory)
+
+
+@pytest.mark.parametrize('uri_prefix, uri_path, expected_path, mtype', [
+    ('/static/', '/css/test.css', '/css/test.css', 'text/css'),
+    ('/static', '/css/test.css', '/css/test.css', 'text/css'),
+    (
+        '/static',
+        '/' + ('t' * StaticRoute._MAX_NON_PREFIXED_LEN),
+        '/' + ('t' * StaticRoute._MAX_NON_PREFIXED_LEN),
+        'application/octet-stream',
+    ),
+    ('/static', '/.test.css', '/.test.css', 'text/css'),
+    ('/some/download/', '/report.pdf', '/report.pdf', 'application/pdf'),
+    ('/some/download/', '/Fancy Report.pdf', '/Fancy Report.pdf', 'application/pdf'),
+    ('/some/download', '/report.zip', '/report.zip', 'application/zip'),
+    ('/some/download', '/foo/../report.zip', '/report.zip', 'application/zip'),
+    ('/some/download', '/foo/../bar/../report.zip', '/report.zip', 'application/zip'),
+    ('/some/download', '/foo/bar/../../report.zip', '/report.zip', 'application/zip'),
+])
+def test_good_path(uri_prefix, uri_path, expected_path, mtype, monkeypatch):
+    monkeypatch.setattr(io, 'open', lambda path, mode: path)
+
+    sr = StaticRoute(uri_prefix, '/var/www/statics')
+
+    req_path = uri_prefix[:-1] if uri_prefix.endswith('/') else uri_prefix
+    req_path += uri_path
+
+    req = Request(testing.create_environ(
+        host='test.com',
+        path=req_path,
+        app='statics'
+    ))
+
+    resp = Response()
+
+    sr(req, resp)
+
+    assert resp.content_type == mtype
+    assert resp.stream == '/var/www/statics' + expected_path
+
+
+def test_lifo(client, monkeypatch):
+    monkeypatch.setattr(io, 'open', lambda path, mode: [path.encode('utf-8')])
+
+    client.app.add_static_route('/downloads', '/opt/somesite/downloads')
+    client.app.add_static_route('/downloads/archive', '/opt/somesite/x')
+
+    response = client.simulate_request(path='/downloads/thing.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.text == '/opt/somesite/downloads/thing.zip'
+
+    response = client.simulate_request(path='/downloads/archive/thingtoo.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.text == '/opt/somesite/x/thingtoo.zip'
+
+
+def test_lifo_negative(client, monkeypatch):
+    monkeypatch.setattr(io, 'open', lambda path, mode: [path.encode('utf-8')])
+
+    client.app.add_static_route('/downloads/archive', '/opt/somesite/x')
+    client.app.add_static_route('/downloads', '/opt/somesite/downloads')
+
+    response = client.simulate_request(path='/downloads/thing.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.text == '/opt/somesite/downloads/thing.zip'
+
+    response = client.simulate_request(path='/downloads/archive/thingtoo.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.text == '/opt/somesite/downloads/archive/thingtoo.zip'
+
+
+def test_downloadable(client, monkeypatch):
+    monkeypatch.setattr(io, 'open', lambda path, mode: [path.encode('utf-8')])
+
+    client.app.add_static_route('/downloads', '/opt/somesite/downloads', downloadable=True)
+    client.app.add_static_route('/assets/', '/opt/somesite/assets')
+
+    response = client.simulate_request(path='/downloads/thing.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.headers['Content-Disposition'] == 'attachment; filename="thing.zip"'
+
+    response = client.simulate_request(path='/downloads/Some Report.zip')
+    assert response.status == falcon.HTTP_200
+    assert response.headers['Content-Disposition'] == 'attachment; filename="Some Report.zip"'
+
+    response = client.simulate_request(path='/assets/css/main.css')
+    assert response.status == falcon.HTTP_200
+    assert 'Content-Disposition' not in response.headers
+
+
+def test_downloadable_not_found(client):
+    client.app.add_static_route('/downloads', '/opt/somesite/downloads', downloadable=True)
+
+    response = client.simulate_request(path='/downloads/thing.zip')
+    assert response.status == falcon.HTTP_404