Repository: tornadoweb/tornado

Issue:
StaticFileHandler returns 304 despite etag mismatch
I am not sure if this is an actual issue in the code or with my deployment, but it caused me some head-scratching.

I am using Tornado as an API webserver for an embedded system. It also serves a small web application using the StaticFileHandler. To fix issues with caching browsers, I subclassed the StaticFileHandler to send Cache-Control headers like so:

    class NoCacheStaticFileHandler(tornado.web.StaticFileHandler):
        def set_extra_headers(self, path):
            self.set_header('Cache-control', 'no-cache, must-revalidate, max-age=0')

The problem I am seeing is that, despite the browser sending a different Etag than the server calculates, the server still sends a 304. I dug a bit into the code and found the following:

    def should_return_304(self):
        """Returns True if the headers indicate that we should return 304.

        .. versionadded:: 3.1
        """
        if self.check_etag_header():
            return True

        # Check the If-Modified-Since, and don't send the result if the
        # content has not been modified
        ims_value = self.request.headers.get("If-Modified-Since")
        if ims_value is not None:
            date_tuple = email.utils.parsedate(ims_value)
            if date_tuple is not None:
                if_since = datetime.datetime(*date_tuple[:6])
                if if_since >= self.modified:
                    return True

        return False

Essentially, this seems to return 304 if either Etag headers match or the If-Modified-Since date the client sent is later than the file on the server. Is this how it should be? The browser sends a different hash, the server knows that the versions do not match but still sends a 304 because the date is newer? It feels like the server should not send a 304 when one property does not match.

The issue might be specific to the system here since the dates there are not very reliable. But I had it several times now that the data on the server was updated but it kept sending 304 despite an Etag mismatch because of the If-Modified-Since headers.


Discussion:
Yeah, I agree that this is probably a bug. When `Etag` is present, it should take priority instead of combining the two headers as the code currently does.

As a workaround, applications deployed in environments without reliable modification timestamps can override `StaticFileHandler.get_modified_time` to return None. This will suppress the `Last-Modified` header in responses, which will in turn (I think) prevent clients from sending `If-Modified-Since`. 
Issue for a quick patch seems to be that the response of `check_etag_header` is a bool so it cannot differentiate between an actual `Etag` check failure and states where the check doesn't apply because the browser did not send an `Etag` header in addition to a successful `Etag` check. I guess making `check_etag_header` return a new type that can represent all cases is not good.

Would it be an option to add a new function to `StaticFileHandler` that just checks if an Etag check is valid. Then modify the first part of `should_return_304` like so:

    if self.etag_header_present():
        return self.check_etag_header()

`etag_header_present` would need to check if the requested site has an Etag value set and if the client sent Etag headers. Essentially, the code would match the beginning of `check_etag_header`:

    def etag_header_present(self):
        computed_etag = utf8(self._headers.get("Etag", ""))
        # Find all weak and strong etag values from If-None-Match header
        # because RFC 7232 allows multiple etag values in a single header.
        etags = re.findall(
            br'\*|(?:W/)?"[^"]*"',
            utf8(self.request.headers.get("If-None-Match", ""))
        )
        if not computed_etag or not etags:
            return False

I can make a PR if you'd approve of that.
Yes, I think something like this is the right approach. But I'd just do it based on the presence of an `If-None-Match` header (if it's there at all, we should use it and ignore `If-Modified-Since`). This may send a full response when `If-Modified-Since` may tell us to send a 304, but when the two validators conflict with each other I think it's better to err on the side of sending a full response. 

Resolving patch:
diff --git a/tornado/web.py b/tornado/web.py
--- a/tornado/web.py
+++ b/tornado/web.py
@@ -2499,8 +2499,9 @@ def should_return_304(self):
 
         .. versionadded:: 3.1
         """
-        if self.check_etag_header():
-            return True
+        # If client sent If-None-Match, use it, ignore If-Modified-Since
+        if self.request.headers.get('If-None-Match'):
+            return self.check_etag_header()
 
         # Check the If-Modified-Since, and don't send the result if the
         # content has not been modified


Test patch:
diff --git a/tornado/test/web_test.py b/tornado/test/web_test.py
--- a/tornado/test/web_test.py
+++ b/tornado/test/web_test.py
@@ -1081,6 +1081,13 @@ def test_static_304_if_none_match(self):
             'If-None-Match': response1.headers['Etag']})
         self.assertEqual(response2.code, 304)
 
+    def test_static_304_etag_modified_bug(self):
+        response1 = self.get_and_head("/static/robots.txt")
+        response2 = self.get_and_head("/static/robots.txt", headers={
+            'If-None-Match': '"MISMATCH"',
+            'If-Modified-Since': response1.headers['Last-Modified']})
+        self.assertEqual(response2.code, 200)
+
     def test_static_if_modified_since_pre_epoch(self):
         # On windows, the functions that work with time_t do not accept
         # negative values, and at least one client (processing.js) seems