Repository: hydroshare/hydroshare

Issue:
cache busting to prevent broken JS and styles on clients
**Describe the feature you'd like and what it will do**
While looking into #4570 I noticed we perhaps don't have any frontend build pipeline.

**Why is this feature important?**
We might consider using a build process on the frontend to increase performance (by minimizing js and css and by setting long retention periods) and also to prevent js/html/css cache issues.

**Is your feature request related to a problem? Please describe.**
Related to #4570



Discussion:
Currently I don't see any cache-control mechanism on production:
    Server: nginx/1.11.13
    Date: Thu, 31 Mar 2022 00:01:30 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 61005
    Connection: keep-alive
    Vary: Cookie, Origin, Accept-Language
    Content-Language: en
    Set-Cookie: csrftoken=pzLNVM9Os92JCzZLJ8BVr1NSTC5s9gFjKx6sKfF6AKB3GgI2ShhNpb4ETcwmp15Q; expires=Thu, 30-Mar-2023 00:01:30 GMT; Max-Age=31449600; Path=/

Looks like recent Etags are set for most of the JS and CSS files
We could consider setting a cache-control policy header
#3621 tackled this at one point but was later removed.  I'm not sure why.
yes looks like django Freshly was creating issues [#3675](https://github.com/hydroshare/hydroshare/issues/3675)
it was removed in [#3678](https://github.com/hydroshare/hydroshare/pull/3678) and looks like the Freshly folks are no longer supporting
We could explore a different solution. I've only ever used build tools to accomplish this but the idea of a Django package is nice.
Looks like even our old version of Django has a native way to handle this that might work:
[manifeststaticfilesstorage](https://docs.djangoproject.com/en/1.10/ref/contrib/staticfiles/#manifeststaticfilesstorage)
@sblack-usu @martinseul is this something you would like me to experiment with?

Resolving patch:
diff --git a/hydroshare/settings.py b/hydroshare/settings.py
--- a/hydroshare/settings.py
+++ b/hydroshare/settings.py
@@ -227,6 +227,8 @@
 # Example: "/home/media/media.lawrence.com/static/"
 STATIC_ROOT = os.path.join(PROJECT_ROOT, STATIC_URL.strip("/"))
 
+STATICFILES_STORAGE = "hydroshare.storage.ForgivingManifestStaticFilesStorage"
+
 # URL that handles the media served from MEDIA_ROOT. Make sure to use a
 # trailing slash.
 # Examples: "http://media.lawrence.com/media/", "http://example.com/media/"
@@ -371,7 +373,10 @@
                 "mezzanine.template.loaders.host_themes.Loader",
                 "django.template.loaders.filesystem.Loader",
                 "django.template.loaders.app_directories.Loader",
-            ]
+            ],
+            'builtins': [
+                'django.templatetags.static',
+            ],
         },
     },
 ]
diff --git a/hydroshare/storage.py b/hydroshare/storage.py
new file mode 100644
--- /dev/null
+++ b/hydroshare/storage.py
@@ -0,0 +1,22 @@
+from django.contrib.staticfiles.storage import ManifestStaticFilesStorage
+import logging
+
+
+class ForgivingManifestStaticFilesStorage(ManifestStaticFilesStorage):
+    """
+    Allow collectstatic to continue even if files are missing for 3rd party libraries
+    Later versions of django have a 'manifest_strict' parameter that would allow this:
+    https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/#django.contrib.staticfiles.storage.ManifestStaticFilesStorage.manifest_strict
+    """
+    logger = logging.getLogger()
+
+    def hashed_name(self, name, content=None, filename=None):
+        try:
+            result = super().hashed_name(name, content, filename)
+        except ValueError as ex:
+            # When the file is missing, let's forgive and ignore that.
+            msg = f"Ignoring ValueError for missing file: {name}, during static collection. \nError: {str(ex)}"
+            print(msg)
+            self.logger.error(msg)
+            result = name
+        return result


Test patch:
diff --git a/theme/templates/tests.html b/theme/templates/tests.html
--- a/theme/templates/tests.html
+++ b/theme/templates/tests.html
@@ -1,5 +1,4 @@
 {% extends "base.html" %}
-{% load static %}
 
 {% block meta_title %}QUnit Test Suite{% endblock %}