Repository: hydroshare/hydroshare

Issue:
cache busting to prevent broken JS and styles on clients
**Describe the feature you'd like and what it will do**
While looking into #4570 I noticed we perhaps don't have any frontend build pipeline.

**Why is this feature important?**
We might consider using a build process on the frontend to increase performance (by minimizing js and css and by setting long retention periods) and also to prevent js/html/css cache issues.

**Is your feature request related to a problem? Please describe.**
Related to #4570



Discussion:
Currently I don't see any cache-control mechanism on production:
    Server: nginx/1.11.13
    Date: Thu, 31 Mar 2022 00:01:30 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 61005
    Connection: keep-alive
    Vary: Cookie, Origin, Accept-Language
    Content-Language: en
    Set-Cookie: csrftoken=pzLNVM9Os92JCzZLJ8BVr1NSTC5s9gFjKx6sKfF6AKB3GgI2ShhNpb4ETcwmp15Q; expires=Thu, 30-Mar-2023 00:01:30 GMT; Max-Age=31449600; Path=/

Looks like recent Etags are set for most of the JS and CSS files
We could consider setting a cache-control policy header
#3621 tackled this at one point but was later removed.  I'm not sure why.
yes looks like django Freshly was creating issues [#3675](https://github.com/hydroshare/hydroshare/issues/3675)
it was removed in [#3678](https://github.com/hydroshare/hydroshare/pull/3678) and looks like the Freshly folks are no longer supporting
We could explore a different solution. I've only ever used build tools to accomplish this but the idea of a Django package is nice.
Looks like even our old version of Django has a native way to handle this that might work:
[manifeststaticfilesstorage](https://docs.djangoproject.com/en/1.10/ref/contrib/staticfiles/#manifeststaticfilesstorage)
@sblack-usu @martinseul is this something you would like me to experiment with?
we could consider [pipline](https://django-pipeline.readthedocs.io) or [whitenoise](https://whitenoise.evans.io/en/stable/) for minification, but I think we should wait until we upgrade our django version before implementing these
we found issues with this during 1.56 deploy, so reopening

Resolving patch:
diff --git a/hydroshare/settings.py b/hydroshare/settings.py
--- a/hydroshare/settings.py
+++ b/hydroshare/settings.py
@@ -227,8 +227,10 @@
 # Example: "/home/media/media.lawrence.com/static/"
 STATIC_ROOT = os.path.join(PROJECT_ROOT, STATIC_URL.strip("/"))
 
-# TODO: Manifest static file storage deactivated for now, pending further debugging
-# STATICFILES_STORAGE = "hydroshare.storage.ForgivingManifestStaticFilesStorage"
+# using this storage class might cause issues for future tests
+# The documentation suggests using the default storage backend when testing
+# https://docs.djangoproject.com/en/1.11/ref/contrib/staticfiles/#django.contrib.staticfiles.storage.ManifestStaticFilesStorage.manifest_strict
+STATICFILES_STORAGE = 'hydroshare.storage.ForgivingManifestStaticFilesStorage'
 
 # URL that handles the media served from MEDIA_ROOT. Make sure to use a
 # trailing slash.
@@ -707,9 +709,6 @@
 SESSION_COOKIE_SECURE = USE_SECURITY
 CSRF_COOKIE_SECURE = USE_SECURITY
 
-# detect test mode to turn off some features
-TESTING = len(sys.argv) > 1 and sys.argv[1] == 'test'
-
 # Categorization in discovery of content types
 # according to file extension of otherwise unaggregated files. 
 DISCOVERY_EXTENSION_CONTENT_TYPES = { 
diff --git a/hydroshare/storage.py b/hydroshare/storage.py
--- a/hydroshare/storage.py
+++ b/hydroshare/storage.py
@@ -4,11 +4,15 @@
 
 class ForgivingManifestStaticFilesStorage(ManifestStaticFilesStorage):
     """
-    Allow collectstatic to continue even if files are missing for 3rd party libraries
-    Later versions of django have a 'manifest_strict' parameter that would allow this:
-    https://docs.djangoproject.com/en/4.0/ref/contrib/staticfiles/#django.contrib.staticfiles.storage.ManifestStaticFilesStorage.manifest_strict
+    Allow collectstatic to continue even if files are missing
     """
-    logger = logging.getLogger()
+    logger = logging.getLogger('django.contrib.staticfiles')
+
+    # Ideally, just setting manifest_strict would solve this issue but there is a known issue:
+    # https://code.djangoproject.com/ticket/31520
+    # So overriding one of the functions
+    # stored_name calls hashed_name so we don't have to override it as well
+    manifest_strict = False
 
     def hashed_name(self, name, content=None, filename=None):
         try:
@@ -17,6 +21,6 @@ def hashed_name(self, name, content=None, filename=None):
             # When the file is missing, let's forgive and ignore that.
             msg = f"Ignoring ValueError for missing file: {name}, during static collection. \nError: {str(ex)}"
             print(msg)
-            self.logger.error(msg)
+            self.logger.warning(msg)
             result = name
         return result


Test patch: