Change:
If you serve files from , nginx will set last-modified header to unix timestamp 1. The result is that browser will cache indefinitely. Not sure what we should do about it really, following nginx configuration helps:
Why do you need to turn etag off?
Ah, I see. nginx's etags aren't hashes.
I wonder if we could find a way to pull the hash out of the nix store path and put it into an etag header.
Thanks for finding this. I've been having issues with broken caching on my nginx/nixos server. Even when the files on the server were updated (files in the nix store), the browser was told that no change had happened. If I'm understanding this correctly, since nginx's etags are based on filesystem modification time; you can't have proper cache invalidation (ie. it will always send 304 not modified). Or is there a workaround?
+1 this is a problem. And turning off etag seems like overkill. Is there some way to 1) set the date on a file so that Nginx can pick it up, or 2) do something else awesome?
Is the zero-timestamp rule set in stone in Nix, or is there some way to override it if you really want a file in /nix/store to have a particular timestamp?
I think we could get away with: It's not perfect, as it will change on each Nix evaluation instead of being content addressed. Another way would be to turn the store path passed to nginx directive to translate into an etag.
The etag function is at called from and . Making it dependent on the realpath of the file seems doable.
Explanation how to configure nginx to use the hash from the nix path:
In my testing, this sends the correct etag, but does not send the 304 not modified, when the etag matches. Also, the header gets stripped out when compression is enabled.
It's working perfectly for me. I've gzip compression enabled, the etag header is sent and the browser gets a 304 when querying the header.
Nice! An example with a directory would be more common:
I've to correct what I said. your statement seems to be correct. What happens in my case is that Cloudflare is actually handling the caching of the ETag correctly, not nginx. So this approach is completely useless without external caching. I'll investigate further and see if I can come up with a solution.
Here is a patch: If real path (after dereferencing symlinks) of is in Nix store, it will use path's hash as ETag. For example: I'll take some time to double check everything and only then send a PR. However, feel free to test: nix { pkgs, ... }: { = { enable = true; package = with pkgs; nginx.overrideAttrs (super: { patches = (super.patches or []) ++ [(fetchpatch { url = ; sha256 = "16pa1vwcm7kibkjsxpk3szaa2vnxdinml6v0fp4038i2qaav113k"; })]; }); }; }
This is really great! or anyone, do you have a moment to write an explanation of this new feature and its practical ramifications for anyone configuring an nginx server? Since it isn't typical nginx behavior, I think the nixos manual should explain it somewhere so it doesn't catch people by surprise.
I think is a good explanation, just needs to be put into the manual.
Cool, submitted .

Discussion:
# verifies: #   1. nginx generates config file with shared http context definitions above #      generated virtual hosts config. <ins> #   2. whether the ETag header is properly generated whenever we're serving #      files in Nix store paths </ins> <del> import ./make-test.nix ({ pkgs, ...} : { </del> <ins> import ./make-test.nix ({ pkgs, ... }: { </ins> name = "nginx"; meta = with pkgs.stdenv.lib.maintainers; { maintainers = [ mbbx6spp ]; }; <del> nodes = { webserver = { ... }: { services.nginx.enable = true; services.nginx.commonHttpConfig = '' </del> <ins> nodes = let commonConfig = { pkgs, ... }: { services.nginx.enable = true; services.nginx.commonHttpConfig = '' </ins> log_format ceeformat '@cee: {"status":"$status",' '"request_time":$request_time,' '"upstream_response_time":$upstream_response_time,'

'"request":"$request",' '"http_referer":"$http_referer",' '"upstream_addr":"$upstream_addr"}'; <ins> ''; services.nginx.virtualHosts."0.my.test" = { extraConfig = '' access_log syslog:server=unix:/dev/log,facility=user,tag=mytag,severity=info ceeformat; location /favicon.ico { allow all; access_log off; log_not_found off; } </ins> ''; <del> services.nginx.virtualHosts."0.my.test" = { extraConfig = '' access_log syslog:server=unix:/dev/log,facility=user,tag=mytag,severity=info ceeformat; location /favicon.ico { allow all; access_log off; log_not_found off; } ''; }; </del> }; <ins> services.nginx.virtualHosts.localhost = { root = pkgs.runCommand "testdir" {} '' mkdir "$out" echo hello world > "$out/index.html" ''; }; }; in { webserver = commonConfig; newwebserver = { pkgs, lib, ... }: { imports = [ commonConfig ]; services.nginx.virtualHosts.localhost = { root = lib.mkForce (pkgs.runCommand "testdir2" {} '' mkdir "$out" echo hello world > "$out/index.html" ''); }; }; </ins> }; <del> testScript = '' startAll; </del> <ins> testScript = { nodes, ... }: let newServerSystem = nodes.newwebserver.config.system.build.toplevel; switch = "${newServerSystem}/bin/switch-to-configuration test"; in '' my $url = 'http://localhost/index.html'; sub checkEtag { my $etag = $webserver->succeed( 'curl -v '.$url.' 2>&1 | sed -n -e "s/^< [Ee][Tt][Aa][Gg]: *//p"' ); $etag =~ s/r?n$//; my $httpCode = $webserver->succeed( 'curl -w "%{http_code}" -X HEAD -H 'If-None-Match: '.$etag.'' '.$url ); chomp $httpCode; die "HTTP code is not 304" unless $httpCode == 304; return $etag; } </ins> $webserver->waitForUnit("nginx"); $webserver->waitForOpenPort("80"); <ins> subtest "check ETag if serving Nix store paths", sub { my $oldEtag = checkEtag; $webserver->succeed('${switch}'); my $newEtag = checkEtag; die "Old ETag $oldEtag is the same as $newEtag" if $oldEtag eq $newEtag; }; </ins> ''; })

{ stdenv, fetchurl, fetchpatch, openssl, zlib, pcre, libxml2, libxslt <del> , gd, geoip </del> <ins> , substituteAll, gd, geoip </ins> , withDebug ? false , withStream ? true , withMail ? false

preConfigure = (concatMapStringsSep "n" (mod: mod.preConfigure or "") modules); <del> patches = stdenv.lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [ </del> <ins> patches = stdenv.lib.singleton (substituteAll { src = ./nix-etag-1.15.4.patch; preInstall = '' export nixStoreDir="$NIX_STORE" nixStoreDirLen="''${#NIX_STORE}" ''; }) ++ stdenv.lib.optionals (stdenv.hostPlatform != stdenv.buildPlatform) [ </ins> (fetchpatch { url = "https://raw.githubusercontent.com/openwrt/packages/master/net/nginx/patches/102-sizeof_test_fix.patch"; sha256 = "0i2k30ac8d7inj9l6bl0684kjglam2f68z8lf3xggcc2i5wzhh8a";

<ins> From f6a978f024d01202f954483423af1b2d5d5159a6 Mon Sep 17 00:00:00 2001 From: Yegor Timoshenko <yegortimoshenko@riseup.net> Date: Fri, 28 Sep 2018 03:27:04 +0000 Subject: [PATCH] If root is in Nix store, set ETag to its path hash --- src/http/ngx_http_core_module.c | 56 +++++++++++++++++++++++++++++---- 1 file changed, 50 insertions(+), 6 deletions(-) diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c index c57ec00c..b7992de2 100644 --- a/src/http/ngx_http_core_module.c +++ b/src/http/ngx_http_core_module.c @@ -1583,6 +1583,7 @@ ngx_http_set_etag(ngx_http_request_t *r) { ngx_table_elt_t           *etag; ngx_http_core_loc_conf_t  *clcf; +    u_char                    *real, *ptr1, *ptr2; clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); @@ -1598,16 +1599,61 @@ ngx_http_set_etag(ngx_http_request_t *r) etag->hash = 1; ngx_str_set(&etag->key, "ETag"); -    etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3); -    if (etag->value.data == NULL) { +    real = ngx_realpath(clcf->root.data, NULL); + +    if (real == NULL) { etag->hash = 0; return NGX_ERROR; } -    etag->value.len = ngx_sprintf(etag->value.data, ""%xT-%xO"", -                                  r->headers_out.last_modified_time, -                                  r->headers_out.content_length_n) -                      - etag->value.data; +    #define NIX_STORE_DIR "@nixStoreDir@" +    #define NIX_STORE_LEN @nixStoreDirLen@ + +    if (r->headers_out.last_modified_time == 1 +        && !ngx_strncmp(real, NIX_STORE_DIR, NIX_STORE_LEN) +        && real[NIX_STORE_LEN] == '/' +        && real[NIX_STORE_LEN + 1] != '0') +    { +        ptr1 = real + NIX_STORE_LEN; +        *ptr1 = '"'; + +        ptr2 = (u_char *) ngx_strchr(ptr1, '-'); + +        if (ptr2 == NULL) { +            ngx_free(real); +            etag->hash = 0; +            return NGX_ERROR; +        } + +        *ptr2++ = '"'; +        *ptr2 = '0'; + +        etag->value.len = ngx_strlen(ptr1); +        etag->value.data = ngx_pnalloc(r->pool, etag->value.len); + +        if (etag->value.data == NULL) { +            ngx_free(real); +            etag->hash = 0; +            return NGX_ERROR; +        } + +        ngx_memcpy(etag->value.data, ptr1, etag->value.len); +    } else { +        etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3); + +        if (etag->value.data == NULL) { +            ngx_free(real); +            etag->hash = 0; +            return NGX_ERROR; +        } + +        etag->value.len = ngx_sprintf(etag->value.data, ""%xT-%xO"", +                                      r->headers_out.last_modified_time, +                                      r->headers_out.content_length_n) +                          - etag->value.data; +    } + +    ngx_free(real); r->headers_out.etag = etag; -- 2.19.0 </ins>