Repository: pypi/warehouse

Issue:
Only trust X-Forwarded-* headers when they come from Fastly
Split out from #248 because it's not exactly related to that.

We need a way to ensure that the `X-Forwarded-*` headers are only trusted when they come from Fastly. Additionally we need to ensure that Fastly is indeed actually setting them and stripping from incoming requests.



Discussion:
On #248 @ewdurbin said:

> short convo with @dstufft on IRC lead to two options on trusting specific headers.
> - maintain a whitelist of upstream Fastly cache nodes
> - generate HTTPS Client Certs for Fastly nodes
> 
> HTTPS Client Certs leave us less need for manually or regularly updating an IP white list. (At this time > Fastly distributes this information via mailing list updates)
> 
> Generating these certs could occur on a per backend basis, one cert for Fastly and multiple for developer and infrastructure use.
> 
> Responsibility for stripping unauthorized headers could fall on either nginx or the warehouse application.
> 
> nginx can be used to verify the client certificate and either set an nginx boolean or a header (with a header we would need to strip it beforehand).

@mattrobenolt said on irc:

```
[00:59:13]  <mattrobenolt>   dstufft: We do the Fastly <-> Disqus verification with a secret header.
[00:59:48]  <mattrobenolt>   So we set a Fastly header when it sends the request to the backend. Just some random key. Then our backends reject anything that doesn't have that key.
```

Specifically, in vcl we do:

```
sub vcl_recv {
  set req.http.X-Forwarded-For = req.http.Fastly-Client-IP;
  set req.http.X-Something-Something = "foo";
}
```

Then in haproxy:

```
acl via_fastly hdr(X-Something-Something) foo
block unless via_fastly
```

Using a secret header seems to be the easiest method to me.

secret header sounds straightforward and easy to manage.

@dstufft where in the stack would you like header validation to exist and how should non-cdn traffic be handled?

@ewdurbin If we can make Nginx strip X-Forwarded-\* headers unless that secret header exists that would be easiest. If that's hard to do (or bad to do) I can do it in a WSGI middleware too. It doesn't matter to me. I don't mind if non cdn traffic can still hit the backend directly (makes it easy for us to test things) we should just strip the headers when it's non CDN.

Can you use this to solve the problems? http://wiki.nginx.org/HttpRealipModule We do this to keep the IP address from the first hop outside datacenter. So using `set_real_ip_from`. I bet you can mix and match to make magic.

## 

Matt Robenolt
@mattrobenolt

On Sunday, April 6, 2014 at 5:23 PM, Donald Stufft wrote:

> @ewdurbin (https://github.com/ewdurbin) If we can make Nginx strip X-Forwarded-\* headers unless that secret header exists that would be easiest. If that's hard to do (or bad to do) I can do it in a WSGI middleware too. It doesn't matter to me. I don't mind if non cdn traffic can still hit the backend directly (makes it easy for us to test things) we should just strip the headers when it's non CDN.
> 
> —
> Reply to this email directly or view it on GitHub (https://github.com/pypa/warehouse/issues/256#issuecomment-39688123).

In other words, it's easy for me to do it in Warehouse, but I have a slight preference to not do it in Warehouse if it's not hard.

Actually, if you just set `set_real_ip_from` to Fastly's IPs as well as your internal subnet, everything should be fine, unless it's possible to spoof the connecting IP address to nginx, and afaik, that's not possible.

in the interest of not maintaining a whitelist for `set_real_ip_from` i'd push for warehouse to have a configurable secret header.

ultimately this pushes the responsibility for verifying the header and stripping the appropriate headers to middleware in the warehouse project.

This regressed when we switched to Pyramid.


Resolving patch:
diff --git a/warehouse/cli/serve.py b/warehouse/cli/serve.py
--- a/warehouse/cli/serve.py
+++ b/warehouse/cli/serve.py
@@ -62,7 +62,6 @@ def serve(config, bind, reload_, workers):
     options = {
         # The gunicorn docs recommend (2 x $num_cores) + 1
         "workers": (2 * multiprocessing.cpu_count()) + 1,
-        "secure_scheme_headers": {"WAREHOUSE-PROTO": "https"},
         "forwarded_allow_ips": "*",
     }
 
diff --git a/warehouse/config.py b/warehouse/config.py
--- a/warehouse/config.py
+++ b/warehouse/config.py
@@ -18,11 +18,12 @@
 import transaction
 
 from pyramid import renderers
-from pyramid.config import Configurator
+from pyramid.config import Configurator as _Configurator
 from pyramid.httpexceptions import HTTPMovedPermanently
 from pyramid.response import Response
 
 from warehouse import __commit__
+from warehouse.utils.proxy import ProxyFixer
 from warehouse.utils.static import WarehouseCacheBuster
 
 
@@ -31,6 +32,24 @@ class Environment(enum.Enum):
     development = "development"
 
 
+class Configurator(_Configurator):
+
+    def add_wsgi_middleware(self, middleware, *args, **kwargs):
+        middlewares = self.get_settings().setdefault("wsgi.middlewares", [])
+        middlewares.append((middleware, args, kwargs))
+
+    def make_wsgi_app(self, *args, **kwargs):
+        # Get the WSGI application from the underlying configurator
+        app = super().make_wsgi_app(*args, **kwargs)
+
+        # Look to see if we have any WSGI middlewares configured.
+        for middleware, args, kw in self.get_settings()["wsgi.middlewares"]:
+            app = middleware(app, *args, **kw)
+
+        # Finally, return our now wrapped app
+        return app
+
+
 def content_security_policy_tween_factory(handler, registry):
     policy = registry.settings.get("csp", {})
     policy = "; ".join([" ".join([k] + v) for k, v in sorted(policy.items())])
@@ -113,6 +132,7 @@ def configure(settings=None):
     )
 
     # Pull in default configuration from the environment.
+    maybe_set(settings, "warehouse.token", "WAREHOUSE_TOKEN")
     maybe_set(settings, "site.name", "SITE_NAME", default="Warehouse")
     maybe_set(settings, "aws.key_id", "AWS_ACCESS_KEY_ID")
     maybe_set(settings, "aws.secret_key", "AWS_SECRET_ACCESS_KEY")
@@ -287,6 +307,13 @@ def configure(settings=None):
         ),
     )
 
+    # Enable support of passing certain values like remote host, client
+    # address, and protocol support in from an outer proxy to the application.
+    config.add_wsgi_middleware(
+        ProxyFixer,
+        token=config.registry.settings["warehouse.token"],
+    )
+
     # Scan everything for configuration
     config.scan(ignore=["warehouse.migrations.env"])
 
diff --git a/warehouse/utils/proxy.py b/warehouse/utils/proxy.py
new file mode 100644
--- /dev/null
+++ b/warehouse/utils/proxy.py
@@ -0,0 +1,51 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import hmac
+
+
+class ProxyFixer:
+
+    def __init__(self, app, token):
+        self.app = app
+        self.token = token
+
+    def __call__(self, environ, start_response):
+        # Determine if the request comes from a trusted proxy or not by looking
+        # for a token in the request.
+        request_token = environ.get("HTTP_WAREHOUSE_TOKEN")
+        if (request_token is not None
+                and hmac.compare_digest(self.token, request_token)):
+            # Compute our values from the environment.
+            proto = environ.get("HTTP_WAREHOUSE_PROTO", "")
+            remote_addr = environ.get("HTTP_WAREHOUSE_IP", "")
+            host = environ.get("HTTP_WAREHOUSE_HOST", "")
+
+            # Put the new header values into our environment.
+            if remote_addr:
+                environ["REMOTE_ADDR"] = remote_addr
+            if host:
+                environ["HTTP_HOST"] = host
+            if proto:
+                environ["wsgi.url_scheme"] = proto
+
+        # Remove any of the forwarded or warehouse headers from the environment
+        for header in {
+                "HTTP_X_FORWARDED_PROTO", "HTTP_X_FORWARDED_FOR",
+                "HTTP_X_FORWARDED_HOST", "HTTP_WAREHOUSE_TOKEN",
+                "HTTP_WAREHOUSE_PROTO", "HTTP_WAREHOUSE_IP",
+                "HTTP_WAREHOUSE_HOST"}:
+            if header in environ:
+                del environ[header]
+
+        # Dispatch to the real underlying application.
+        return self.app(environ, start_response)


Test patch:
diff --git a/tests/conftest.py b/tests/conftest.py
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -92,6 +92,7 @@ def drop_database():
 def app_config(database):
     config = configure(
         settings={
+            "warehouse.token": "insecure token",
             "camo.url": "http://localhost:9000/",
             "camo.key": "insecure key",
             "database.url": database,
diff --git a/tests/unit/cli/test_serve.py b/tests/unit/cli/test_serve.py
--- a/tests/unit/cli/test_serve.py
+++ b/tests/unit/cli/test_serve.py
@@ -69,7 +69,6 @@ def test_serve(monkeypatch, cli):
                 "reload": False,
                 "workers": 3,
                 "proc_name": "warehouse",
-                "secure_scheme_headers": {"WAREHOUSE-PROTO": "https"},
                 "forwarded_allow_ips": "*",
             },
         ),
diff --git a/tests/unit/test_config.py b/tests/unit/test_config.py
--- a/tests/unit/test_config.py
+++ b/tests/unit/test_config.py
@@ -20,6 +20,7 @@
 from pyramid.httpexceptions import HTTPMovedPermanently
 
 from warehouse import config
+from warehouse.utils.proxy import ProxyFixer
 
 
 class TestCSPTween:
@@ -223,6 +224,7 @@ def test_configure(monkeypatch, settings, environment):
     class FakeRegistry(dict):
         def __init__(self):
             self.settings = {
+                "warehouse.token": "insecure token",
                 "warehouse.env": environment,
                 "camo.url": "http://camo.example.com/",
                 "pyramid.reload_assets": False,
@@ -233,6 +235,7 @@ def __init__(self):
     configurator_obj = pretend.stub(
         registry=FakeRegistry(),
         include=pretend.call_recorder(lambda include: None),
+        add_wsgi_middleware=pretend.call_recorder(lambda m, *a, **kw: None),
         add_renderer=pretend.call_recorder(lambda name, renderer: None),
         add_jinja2_renderer=pretend.call_recorder(lambda renderer: None),
         add_jinja2_search_path=pretend.call_recorder(lambda path, name: None),
@@ -303,6 +306,9 @@ def __init__(self):
 
     assert configurator_cls.calls == [pretend.call(settings=expected_settings)]
     assert result is configurator_obj
+    assert configurator_obj.add_wsgi_middleware.calls == [
+        pretend.call(ProxyFixer, token="insecure token"),
+    ]
     assert configurator_obj.include.calls == (
         [
             pretend.call(x) for x in [
diff --git a/tests/unit/utils/test_proxy.py b/tests/unit/utils/test_proxy.py
new file mode 100644
--- /dev/null
+++ b/tests/unit/utils/test_proxy.py
@@ -0,0 +1,92 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import pretend
+
+from warehouse.utils import proxy
+
+
+class TestProxyFixer:
+
+    def test_cleans_environ_forwarded(self):
+        response = pretend.stub()
+        app = pretend.call_recorder(lambda e, s: response)
+
+        environ = {
+            "HTTP_X_FORWARDED_PROTO": "http",
+            "HTTP_X_FORWARDED_FOR": "1.2.3.4",
+            "HTTP_X_FORWARDED_HOST": "example.com",
+            "HTTP_SOME_OTHER_HEADER": "woop",
+        }
+        start_response = pretend.stub()
+
+        resp = proxy.ProxyFixer(app, token=None)(environ, start_response)
+
+        assert resp is response
+        assert app.calls == [
+            pretend.call({"HTTP_SOME_OTHER_HEADER": "woop"}, start_response),
+        ]
+
+    def test_skips_headers(self):
+        response = pretend.stub()
+        app = pretend.call_recorder(lambda e, s: response)
+
+        environ = {
+            "HTTP_WAREHOUSE_TOKEN": "NOPE",
+            "HTTP_WAREHOUSE_PROTO": "http",
+            "HTTP_WAREHOUSE_IP": "1.2.3.4",
+            "HTTP_WAREHOUSE_HOST": "example.com",
+        }
+        start_response = pretend.stub()
+
+        resp = proxy.ProxyFixer(app, token="1234")(environ, start_response)
+
+        assert resp is response
+        assert app.calls == [pretend.call({}, start_response)]
+
+    def test_accepts_headers(self):
+        response = pretend.stub()
+        app = pretend.call_recorder(lambda e, s: response)
+
+        environ = {
+            "HTTP_WAREHOUSE_TOKEN": "1234",
+            "HTTP_WAREHOUSE_PROTO": "http",
+            "HTTP_WAREHOUSE_IP": "1.2.3.4",
+            "HTTP_WAREHOUSE_HOST": "example.com",
+        }
+        start_response = pretend.stub()
+
+        resp = proxy.ProxyFixer(app, token="1234")(environ, start_response)
+
+        assert resp is response
+        assert app.calls == [
+            pretend.call(
+                {
+                    "REMOTE_ADDR": "1.2.3.4",
+                    "HTTP_HOST": "example.com",
+                    "wsgi.url_scheme": "http",
+                },
+                start_response,
+            ),
+        ]
+
+    def test_missing_headers(self):
+        response = pretend.stub()
+        app = pretend.call_recorder(lambda e, s: response)
+
+        environ = {"HTTP_WAREHOUSE_TOKEN": "1234"}
+        start_response = pretend.stub()
+
+        resp = proxy.ProxyFixer(app, token="1234")(environ, start_response)
+
+        assert resp is response
+        assert app.calls == [pretend.call({}, start_response)]